-- Logs begin at Wed 2022-07-20 18:36:39 UTC, end at Wed 2022-07-20 19:48:05 UTC. -- Jul 20 19:13:08.479718 dv-cinder-850548-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 19:13:08.492146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 19:13:08 2022] *** Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: nodename: dv-cinder-850548-1-iscsi Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: machine: x86_64 Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: clock source: unix Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: pcre jit disabled Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: detected number of CPU cores: 4 Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: current working directory: / Jul 20 19:13:08.493146 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: detected binary path: /usr/bin/uwsgi-core Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: your processes number limit is 47680 Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: your memory page size is 4096 bytes Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: detected max file descriptor number: 2048 Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: lock engine: pthread robust mutexes Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: thunder lock: enabled Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 19:13:08.493783 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 19:13:08.536646 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: Python main interpreter initialized at 0x56392013b9c0 Jul 20 19:13:08.536646 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: python threads support enabled Jul 20 19:13:08.536646 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: your server socket listen backlog is limited to 100 connections Jul 20 19:13:08.536646 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: your mercy for graceful operations on workers is 90 seconds Jul 20 19:13:08.537922 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 19:13:08.537922 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: *** Operational MODE: preforking *** Jul 20 19:13:08.537922 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: *** uWSGI is running in multiple interpreter mode *** Jul 20 19:13:08.537922 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: spawned uWSGI master process (pid: 80239) Jul 20 19:13:08.538074 dv-cinder-850548-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 19:13:08.539840 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: spawned uWSGI worker 1 (pid: 80249, cores: 1) Jul 20 19:13:08.539840 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: spawned uWSGI worker 2 (pid: 80250, cores: 1) Jul 20 19:13:08.539840 dv-cinder-850548-1-iscsi devstack@keystone.service[80239]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 19:13:14.026586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:13:14.026586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: from cryptography.utils import int_from_bytes Jul 20 19:13:14.027356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:13:14.027356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes Jul 20 19:13:14.027811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:13:14.027811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: from cryptography.utils import int_from_bytes Jul 20 19:13:14.027956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:13:14.027956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes Jul 20 19:13:14.076444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 19:13:14.076654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 19:13:14.076872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] command line args: [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 19:13:14.077177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 19:13:14.077375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ================================================================================ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 19:13:14.077466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 19:13:14.077649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] admin_token = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.077732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 19:13:14.077889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] command line args: [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 19:13:14.077982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config_dir = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.078186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.078288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 19:13:14.078368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] config_source = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.078444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ================================================================================ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 19:13:14.078555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] debug = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.078632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.078775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_dir = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_source = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] debug = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_config_append = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.080952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_dir = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_config_append = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_dir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.081837 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.082903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.083031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.083146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.083258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.083874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.084897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.085906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] publish_errors = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] publish_errors = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.086995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] standard_threads = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.087965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_journal = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_json = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] standard_threads = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_stderr = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] use_syslog = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.088867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.089005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.089127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.089246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.089370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_journal = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.089483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.089594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_json = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.089594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_stderr = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.089809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.089928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_syslog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.090041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:13:14.090271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.090965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.091998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.092946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.093215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.093342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.093456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.093648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.093771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.093890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.094935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.095975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.096934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.097992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.098899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.099010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.099749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.099872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.099984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.100664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.101952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.102929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.103924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.104984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.105918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.106950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.107937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.108961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.109993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.110977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.111917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.112950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.113896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.external = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.114969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.password = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] auth.token = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.external = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.115963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.password = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.116998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.token = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.117899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.118957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.119980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.120945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.121989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.122912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.123964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.124956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.125884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.126927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127877 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.127990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.128953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129769 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.129994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.130107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.130228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.130417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.130547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.130662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.130777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.131961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.132900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.133963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.134881 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.135914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.136830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.137934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.138890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.139980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.140932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.141955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.142989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.143905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.144946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.145764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.146392 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.146548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.146927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.147999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.148857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.149935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.150976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.151888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.152943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.153986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.154902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.155938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.156853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157366 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.157944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.158973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.159885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.160964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.161081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.161196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.161309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.161423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.161580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.162976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.163906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.164990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80250) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:13:14.165562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 19:13:14.196095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.196295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.196379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.196454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.196526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.196655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.204840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.205173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.232981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.233210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.233304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.233528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.378064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.379829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.427519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.427748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.430017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.430256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.493220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.493448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.493675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.493894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.494111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.494326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.494452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.494685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.494908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.495126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.495341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.495554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.520443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.520697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.521051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.521759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.521849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.521977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.522866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.523079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.534040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.534735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.546846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.547955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.548039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.548169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.548252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.548387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80250) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:13:14.566306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.566482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.566567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:13:14.566643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:13:14.566815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.566903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.567014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:13:14.567099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:13:14.567280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.567364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.567534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.567619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.568306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.568411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.568603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.568954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.579894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.580152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.580237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.580426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.580509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.580690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.580774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.581002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.589436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.590029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.593230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.593416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.593503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.593692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.593776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.593973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.594979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.595974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.596054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.596239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.622866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.623082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.624480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.624685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.624787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.624989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.630105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.630758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.631045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.631198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.631353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.631496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.638571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.638661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.638928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.639037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.643516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.643603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.643799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.643882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.644996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.645178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.645259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.645440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.645536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.664836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.665009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.665132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.665282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.669685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.669917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.673350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.673621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.673706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.673914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.678692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.678963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.682315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.682581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.686113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.686451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.686579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.686682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.686783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.686966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.687094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.687255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.687376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.687529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.687639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.687802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.707322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.707469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.707699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.707852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.707967 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.708119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.708227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.709093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.716723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.717045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.718466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.718737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.718820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.719021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.721791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.722071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.722155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.722349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.722432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.722624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.722719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.722900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.722983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.723230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.736445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.736721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.737318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.737953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.738142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.738231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.738428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.738701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.758008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.759098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.761291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.762420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.764682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.765745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.766298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.767362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.767905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.768182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.768469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.769031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.769311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.769597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.776114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.777194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.777757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.779070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.781227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.781510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.781786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.782061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.782326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.782613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.782896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.783171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.791954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.792235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.792510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.793316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.793602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.793878 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.800336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80250) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.800613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.800910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.801193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.801459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.801741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.801825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:13:14.802011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.802177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.802317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.802427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.802605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.802806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.802953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.803929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.804063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.804204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.804327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80250) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.804476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.804736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.805049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.805318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:13:14.845574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56392013b9c0 pid: 80250 (default app) Jul 20 19:13:14.849088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407bd1f-9bac-4937-9954-e5a239924f4c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:14.850082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407bd1f-9bac-4937-9954-e5a239924f4c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:14.850386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407bd1f-9bac-4937-9954-e5a239924f4c None None] PATH_INFO: `/v3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:14.852687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56392013b9c0 pid: 80249 (default app) Jul 20 19:13:14.854575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1/1] 10.222.0.79 () {50 vars in 791 bytes} [Wed Jul 20 19:13:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:22.531294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:22.532126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:22.532126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:22.536874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1/2] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:22.542532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:22.543012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:22.543103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:22.615842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: INFO dbcounter [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] Registered counter for database keystone Jul 20 19:13:22.637689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG oslo_db.sqlalchemy.engines [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80250) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 19:13:22.644273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writer thread running {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 19:13:22.824441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG passlib.handlers.bcrypt [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80250) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 19:13:22.824869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG passlib.handlers.bcrypt [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80250) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 19:13:22.881024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:22.882705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:23.014093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 2/3] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 19:13:23.217638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:23.288219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: INFO dbcounter [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] Registered counter for database keystone Jul 20 19:13:23.310508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG oslo_db.sqlalchemy.engines [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 19:13:23.316630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writer thread running {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 19:13:23.435298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079030f-ad5b-4920-9ee8-9596d79d7d7f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:23.439545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:23.440213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:23.440500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:23.440778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:23.449685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:23.501908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 19:13:23.502195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44809c2-f0ca-4c1f-8954-c6d94217fcc6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:23.514114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 2/4] 10.222.0.79 () {58 vars in 1324 bytes} [Wed Jul 20 19:13:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:13:23.520080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:23.535177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e270d-5e9c-47c9-9c4f-a57b0f11e362 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:23.540071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:23.541252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:23.541553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:23.541833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] PATH_INFO: `/v3/projects/8afe543881644c988270ab8de48728b1` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:23.545943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] RBAC: Authorizing `identity:get_project(project_id=8afe543881644c988270ab8de48728b1)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:23.597775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 19:13:23.598118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d019692-7a26-431f-952d-5b0aabcfdaa3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:23.603793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 3/5] 10.222.0.79 () {56 vars in 1207 bytes} [Wed Jul 20 19:13:23 2022] GET /identity/v3/projects/8afe543881644c988270ab8de48728b1 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:25.676312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0d21-b0c3-484a-b53f-83e0fe740046 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.676921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0d21-b0c3-484a-b53f-83e0fe740046 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.676921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0d21-b0c3-484a-b53f-83e0fe740046 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.681069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 3/6] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:25.686895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.687385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.687473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.708902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:25.710501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:25.719076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 4/7] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 19:13:25.741999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0d21-b0c3-484a-b53f-83e0fe740046 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:25.758875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0d21-b0c3-484a-b53f-83e0fe740046 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:25.765007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce8c9b-3534-411e-a849-2f2d90d2e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:25.765660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce8c9b-3534-411e-a849-2f2d90d2e56c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.765948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce8c9b-3534-411e-a849-2f2d90d2e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.766228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce8c9b-3534-411e-a849-2f2d90d2e56c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.776294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce8c9b-3534-411e-a849-2f2d90d2e56c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:25.778930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce8c9b-3534-411e-a849-2f2d90d2e56c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:25.791770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 4/8] 10.222.0.79 () {58 vars in 1324 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:13:25.798104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:25.809547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2df8-a5a6-4e61-8aad-9134f2261221 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:25.815160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e08db-3ba5-4cd1-9844-f426d99c7bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:25.816545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e08db-3ba5-4cd1-9844-f426d99c7bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.817051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e08db-3ba5-4cd1-9844-f426d99c7bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.817532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e08db-3ba5-4cd1-9844-f426d99c7bbb admin admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.819821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e08db-3ba5-4cd1-9844-f426d99c7bbb admin admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:25.823061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e08db-3ba5-4cd1-9844-f426d99c7bbb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:25.824834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 5/9] 10.222.0.79 () {56 vars in 1198 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:27.960018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0561a-618a-40a4-b94e-058785c01d60 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.960532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0561a-618a-40a4-b94e-058785c01d60 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.960532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0561a-618a-40a4-b94e-058785c01d60 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.961873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 5/10] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:27.967189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.967606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.967879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.984863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:27.986062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:27.995419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed0c46-35b1-4609-abbb-d1f6006e5bbf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.995710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed0c46-35b1-4609-abbb-d1f6006e5bbf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.995990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed0c46-35b1-4609-abbb-d1f6006e5bbf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.997366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 6/11] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:28.002334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.002621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.002902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.043440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80249) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 19:13:28.043861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80249) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 19:13:28.071134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 6/12] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:28.090451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:28.092036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:28.094594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.118302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.198287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 7/13] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:28.200037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99436cd-7f79-4efc-ae36-3d2278b05794 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.221015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.234212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.235020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.235312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.235593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.246852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:28.258938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:28.259856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:28.260808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 7/14] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:28.265903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.322748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ef013-90e3-42be-8c11-a997e438bdc9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.325020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.347557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.361699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.362595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.362880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.363154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.367429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e0280-7b9e-427b-bd75-60e23c7facbb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.402680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.403598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.403956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.404170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.404956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:28.405269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.406685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.419369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 8/15] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:28.424996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.431212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.432730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.524705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9225a91-8abd-4dc6-8ca4-e75feda248a0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.532701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 8/16] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:13:28.539015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.541758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.559834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.560715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.561123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.561410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.562611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.564072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.565733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 9/17] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:28.571309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.587538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.641290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211833a2-c05d-40ea-816d-7b30163f5002 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.675860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.676893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.677283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.677495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.678827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6510aa-3ff5-4e19-b5a7-f0f46ed7868d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.698064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.699550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.718489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.719295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.719585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.719866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:28.721060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:28.721990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:28.727746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:28.727746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:28.727746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:28.727746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:28.728030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.728127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 9/18] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:28.728206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.728206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 10/19] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:28.729539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.729916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:28.787434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.810200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:28.830029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cfab3-81bb-4646-8155-0297f8fc8b02 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.830605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e55f31-e31a-45bf-aa7a-fa09496c3846 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:28.864215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.864608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:28.864906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.865088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:28.865229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.865411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:28.865504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.865695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:28.866332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.domain.id=default, user.id=0b2e9296115548cda14048313e8cf43e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:28.866481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:28.866647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.866797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:28.868141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa27d-f78a-4f0f-9c21-b09a9483893a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.868251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde715df-2535-4916-95c1-f80ac47d8964 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:28.875462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 11/20] 10.222.0.79 () {56 vars in 1322 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/role_assignments?user.id=0b2e9296115548cda14048313e8cf43e&scope.domain.id=default&role.id=689534f62373409f9d02ea4acc335ab7 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:28.887079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 10/21] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:31.255476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55611748-1068-48ff-a53a-36cc62a1f111 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.256546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55611748-1068-48ff-a53a-36cc62a1f111 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.257180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55611748-1068-48ff-a53a-36cc62a1f111 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.258774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 12/22] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:31.264176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f43874-e019-47c0-b211-6e905db0741f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.264593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f43874-e019-47c0-b211-6e905db0741f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.264769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f43874-e019-47c0-b211-6e905db0741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.290852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-52f43874-e019-47c0-b211-6e905db0741f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:31.292055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-52f43874-e019-47c0-b211-6e905db0741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:31.389136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 11/23] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:31.426953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55611748-1068-48ff-a53a-36cc62a1f111 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.487623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cfe08-3efd-470f-84e2-9a7a936dafa3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.487985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cfe08-3efd-470f-84e2-9a7a936dafa3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.488191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cfe08-3efd-470f-84e2-9a7a936dafa3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.490495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 12/24] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:31.496143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ec5a-e6cb-4fae-a602-e28fab47a78e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.496433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ec5a-e6cb-4fae-a602-e28fab47a78e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.496713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ec5a-e6cb-4fae-a602-e28fab47a78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.529291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-20e5ec5a-e6cb-4fae-a602-e28fab47a78e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:31.530432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-20e5ec5a-e6cb-4fae-a602-e28fab47a78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:31.573758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55611748-1068-48ff-a53a-36cc62a1f111 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:31.614268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:31.621118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e3528-23c4-4b2f-b888-514ca881b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:31.621930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e3528-23c4-4b2f-b888-514ca881b220 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.622214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e3528-23c4-4b2f-b888-514ca881b220 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.622593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e3528-23c4-4b2f-b888-514ca881b220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.648380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e3528-23c4-4b2f-b888-514ca881b220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5f0e3528-23c4-4b2f-b888-514ca881b220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:31.659760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:31.660613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:31.662001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:31.668912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 13/25] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:31.669796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7953-ecd4-48c7-bb93-94a79beebbf9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.670096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7953-ecd4-48c7-bb93-94a79beebbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.670374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7953-ecd4-48c7-bb93-94a79beebbf9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.671974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 14/26] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:31.672685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7953-ecd4-48c7-bb93-94a79beebbf9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.675655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 13/27] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:31.683169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.683461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.683742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.713227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:31.724079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:31.725321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:31.866352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 14/28] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:31.867328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.878153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7953-ecd4-48c7-bb93-94a79beebbf9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:31.933575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:31.934256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.934540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.934815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.935572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:31.935879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:31.937294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:31.963129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 15/29] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:31.963979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.050415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cdec8-6689-44c0-9108-11f18efbf17e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.054707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.131667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c28a6f-0e1e-4718-9afa-921b3a2419e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.132477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c28a6f-0e1e-4718-9afa-921b3a2419e0 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.133112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c28a6f-0e1e-4718-9afa-921b3a2419e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.133403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c28a6f-0e1e-4718-9afa-921b3a2419e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.134170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c28a6f-0e1e-4718-9afa-921b3a2419e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.136020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c28a6f-0e1e-4718-9afa-921b3a2419e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.139359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.148842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ea2af-fd3f-4dcc-afea-63ee1e7b8516 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.190654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.191607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.192018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.192861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.204452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.206111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:32.225112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:32.225930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:32.226728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:32.226728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:32.226728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:32.226728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:32.226728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 16/30] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:32.227775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.272610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 15/31] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Jul 20 19:13:31 2022] POST /identity/v3/roles => generated 219 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:32.274321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70691383-4933-45a3-a871-c92f38d2c771 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.274670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70691383-4933-45a3-a871-c92f38d2c771 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.274958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70691383-4933-45a3-a871-c92f38d2c771 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.276570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 16/32] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:32.277335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70691383-4933-45a3-a871-c92f38d2c771 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.387587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.399677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684f7f8-96fb-4caf-8644-b47e21113f5f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.463595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa619382-39db-4835-a807-a338d37ac113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.464500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa619382-39db-4835-a807-a338d37ac113 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.464793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa619382-39db-4835-a807-a338d37ac113 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.465099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa619382-39db-4835-a807-a338d37ac113 None admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.466711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa619382-39db-4835-a807-a338d37ac113 None admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.468754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa619382-39db-4835-a807-a338d37ac113 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.470453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 17/33] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:32.472241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.472535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.473006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.495242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:32.496459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:32.509378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70691383-4933-45a3-a871-c92f38d2c771 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.556646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.557391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.557681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.557996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.558802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:32.559155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.561335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba211fb-5088-4066-a6a2-ae5708f92d0f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.581572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 18/34] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:32.583535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.584348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 17/35] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:32.585658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0546b6-0638-4000-a2ff-9846342d41f4 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.585953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0546b6-0638-4000-a2ff-9846342d41f4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.586236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0546b6-0638-4000-a2ff-9846342d41f4 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.592466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 18/36] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:32.596485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0546b6-0638-4000-a2ff-9846342d41f4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.665361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.697685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.712046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655fdab-e6fb-4272-a89d-54771fedd631 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.722641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0546b6-0638-4000-a2ff-9846342d41f4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.751200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ac89c-29bd-458e-9028-4b8602155337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.752022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ac89c-29bd-458e-9028-4b8602155337 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.752300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ac89c-29bd-458e-9028-4b8602155337 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.752580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ac89c-29bd-458e-9028-4b8602155337 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.754144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ac89c-29bd-458e-9028-4b8602155337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.755991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ac89c-29bd-458e-9028-4b8602155337 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.762725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 19/37] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:32.762725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.762725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.762725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.763927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.764599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.764904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.765183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.765911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.767380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.780359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:32.781844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:32.797430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 19/38] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:13:32 2022] POST /identity/v3/projects => generated 301 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:32.798335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.876585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 20/39] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:32.877491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.939145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f3997-5f81-4cba-9b90-1c950160e45c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.947650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.983655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.984346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.984640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.984956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.985729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.987144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.991376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:33.004416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 20/40] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Jul 20 19:13:32 2022] POST /identity/v3/roles => generated 225 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:33.006436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:33.015887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a5752-bd31-40f3-972f-b05039bb7669 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:33.059410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675ed9c-d15a-4a3e-ae78-f3a79fba027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:33.060720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ed9c-d15a-4a3e-ae78-f3a79fba027e None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:33.061249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ed9c-d15a-4a3e-ae78-f3a79fba027e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:33.061745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ed9c-d15a-4a3e-ae78-f3a79fba027e None admin] PATH_INFO: `/v3/domains/default/users/0b2e9296115548cda14048313e8cf43e/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:33.064279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675ed9c-d15a-4a3e-ae78-f3a79fba027e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0b2e9296115548cda14048313e8cf43e, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:33.066114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675ed9c-d15a-4a3e-ae78-f3a79fba027e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:33.135849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874553-ccae-49f5-9d56-bef339c8fcf4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:33.176087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:33.177342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:33.177836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:33.178119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:33.178886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:33.180287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:33.189984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 21/41] 10.222.0.79 () {58 vars in 1361 bytes} [Wed Jul 20 19:13:32 2022] PUT /identity/v3/domains/default/users/0b2e9296115548cda14048313e8cf43e/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:33.191724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa962-5cba-4dfa-a56a-add0edfe8af1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:33.192713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa962-5cba-4dfa-a56a-add0edfe8af1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:33.193079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa962-5cba-4dfa-a56a-add0edfe8af1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:33.198152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 22/42] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:33.202675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 21/43] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Jul 20 19:13:33 2022] POST /identity/v3/roles => generated 223 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:33.205443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:33.205750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:33.206097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:33.231095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:33.233342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:33.333182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 23/44] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:33.358784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:33.394247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:33.588553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb7ff4-9d38-4843-b944-934853f38930 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:33.623046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579db71-a4fd-442c-999e-12214384bcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:33.623853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579db71-a4fd-442c-999e-12214384bcef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:33.624143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579db71-a4fd-442c-999e-12214384bcef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:33.624425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579db71-a4fd-442c-999e-12214384bcef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:33.625639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579db71-a4fd-442c-999e-12214384bcef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:33.627103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579db71-a4fd-442c-999e-12214384bcef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:33.628985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 22/45] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:33.634502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:33.714462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:33.738622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164ea71-5efa-4826-a013-3bb6fc941bcc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:33.772833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88282ae2-e258-4c4e-aa59-a8ae3f051351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:33.773577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88282ae2-e258-4c4e-aa59-a8ae3f051351 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:33.773868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88282ae2-e258-4c4e-aa59-a8ae3f051351 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:33.774192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88282ae2-e258-4c4e-aa59-a8ae3f051351 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:33.774920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88282ae2-e258-4c4e-aa59-a8ae3f051351 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:33.776500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88282ae2-e258-4c4e-aa59-a8ae3f051351 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:33.804364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 24/46] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:13:33 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:13:35.270255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653d02-0865-4a62-bf1f-7c59deefba79 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.270780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653d02-0865-4a62-bf1f-7c59deefba79 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.270914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653d02-0865-4a62-bf1f-7c59deefba79 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.272306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 23/47] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:35.277493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.277901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.278466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.297598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:35.298784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:35.369432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 25/48] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:35.392394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c653d02-0865-4a62-bf1f-7c59deefba79 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.487164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.494801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c653d02-0865-4a62-bf1f-7c59deefba79 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.529721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.530450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.530734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.531010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.541263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:35.549037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:35.550088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:35.551055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:35.553025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 24/49] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:35.556244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.614771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.657846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c15849-9ca9-4394-968f-ffccaf59927a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.693283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.693961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.694270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.694549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.695322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:35.695632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.697065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:35.706213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 26/50] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:35.713053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.812671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512af63-182a-4de0-9c81-54b0f6745595 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.847239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635d570-1ce5-472d-a737-3166181b9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.848162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635d570-1ce5-472d-a737-3166181b9572 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.848985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635d570-1ce5-472d-a737-3166181b9572 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.849270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635d570-1ce5-472d-a737-3166181b9572 None admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.850470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635d570-1ce5-472d-a737-3166181b9572 None admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.851927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635d570-1ce5-472d-a737-3166181b9572 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:35.853613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 25/51] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:35.859091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.862120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551b247-53eb-42b7-a4ce-10bb91012af2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.862414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551b247-53eb-42b7-a4ce-10bb91012af2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.862706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551b247-53eb-42b7-a4ce-10bb91012af2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.864123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 26/52] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:35.869434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.869729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.870012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.887841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:35.889484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:35.909292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.966262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 27/53] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:35.991277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.071202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e533616-c202-4956-8a7e-8b0793d786ff None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.090751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:36.093057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae3bbf-6052-454e-ad52-dd9ee1a20a25 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.106355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.107258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.107563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.107850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.109061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.110527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.112162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 27/54] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:36.119072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.127823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.128636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.128943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.129275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.130483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.131961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.133664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 28/55] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:36.139092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.161808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:36.233332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eee2f6-d7b8-4294-80e8-e5360e5a1a79 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.244726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c32a8-354e-441c-9f7e-c103b310f8ce None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.267159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.267831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.268123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.268412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.269290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.domain.id=default, user.id=0b2e9296115548cda14048313e8cf43e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:36.269619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.271491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfd1e3-25a5-4141-a02e-1dbab38a4b50 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.278619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc615ec-01aa-4d87-b823-8d4a235714e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.279285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc615ec-01aa-4d87-b823-8d4a235714e0 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.279573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc615ec-01aa-4d87-b823-8d4a235714e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.279853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc615ec-01aa-4d87-b823-8d4a235714e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.280361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 28/56] 10.222.0.79 () {56 vars in 1322 bytes} [Wed Jul 20 19:13:36 2022] GET /identity/v3/role_assignments?user.id=0b2e9296115548cda14048313e8cf43e&scope.domain.id=default&role.id=689534f62373409f9d02ea4acc335ab7 => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:36.280735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc615ec-01aa-4d87-b823-8d4a235714e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.282039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc615ec-01aa-4d87-b823-8d4a235714e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.312574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 29/57] 10.222.0.79 () {60 vars in 1140 bytes} [Wed Jul 20 19:13:36 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:13:38.371029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ffd98-6856-466e-ba44-7f9d49073bf3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:38.371550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ffd98-6856-466e-ba44-7f9d49073bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:38.371660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ffd98-6856-466e-ba44-7f9d49073bf3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:38.373169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 29/58] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:38.378327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:38.378730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:38.378901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:38.397048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:38.398252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:38.464688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 30/59] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:38.488845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ffd98-6856-466e-ba44-7f9d49073bf3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:38.554271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:38.600946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ffd98-6856-466e-ba44-7f9d49073bf3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:38.633695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e5ec9-e593-4112-9670-9fee2ab33649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:38.634413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5ec9-e593-4112-9670-9fee2ab33649 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:38.634701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5ec9-e593-4112-9670-9fee2ab33649 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:38.634980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5ec9-e593-4112-9670-9fee2ab33649 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:38.636132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e5ec9-e593-4112-9670-9fee2ab33649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:38.637655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e5ec9-e593-4112-9670-9fee2ab33649 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:38.639330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 30/60] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:38.644937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:38.647638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:38.749266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941123c4-33df-481a-a257-ec9321d1db9e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:38.784216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60454d-4e8f-407b-8e23-e3a6fce1cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:38.784913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60454d-4e8f-407b-8e23-e3a6fce1cfaa None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:38.785208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60454d-4e8f-407b-8e23-e3a6fce1cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:38.785491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60454d-4e8f-407b-8e23-e3a6fce1cfaa None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:38.786522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60454d-4e8f-407b-8e23-e3a6fce1cfaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:38.787985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60454d-4e8f-407b-8e23-e3a6fce1cfaa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:38.812971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 31/61] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:13:38 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:41.972132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0c724-f7c1-4a8e-8c69-337971a22a20 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.973168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0c724-f7c1-4a8e-8c69-337971a22a20 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.973706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0c724-f7c1-4a8e-8c69-337971a22a20 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:41.975325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 31/62] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:41.980756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b580-3c15-494c-b317-511630e0f1ef None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.981648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b580-3c15-494c-b317-511630e0f1ef None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.982129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b580-3c15-494c-b317-511630e0f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.000207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-33d7b580-3c15-494c-b317-511630e0f1ef None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:42.001795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-33d7b580-3c15-494c-b317-511630e0f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:42.058392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:42.074469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 32/63] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:42.097802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0c724-f7c1-4a8e-8c69-337971a22a20 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.109977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec21e32-04cd-4120-93f9-7f0cc284f40d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.110285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec21e32-04cd-4120-93f9-7f0cc284f40d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.110544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec21e32-04cd-4120-93f9-7f0cc284f40d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.112195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 33/64] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:42.118053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cebbc9-acc8-46f4-b032-4ad7f45e185b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.118345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cebbc9-acc8-46f4-b032-4ad7f45e185b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.118628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cebbc9-acc8-46f4-b032-4ad7f45e185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.153374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d8cebbc9-acc8-46f4-b032-4ad7f45e185b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:42.154579 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d8cebbc9-acc8-46f4-b032-4ad7f45e185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:42.214244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0c724-f7c1-4a8e-8c69-337971a22a20 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.237697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:42.253298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.254484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.255014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.255490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.268918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:42.278710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:42.279566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:42.280330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:42.282704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 32/65] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:42.289386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.323791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 34/66] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:42.353394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa705e3-95b2-40ba-abac-d05b7856c5e6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.354158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa705e3-95b2-40ba-abac-d05b7856c5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.354950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa705e3-95b2-40ba-abac-d05b7856c5e6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.356570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 35/67] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:42.361164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa705e3-95b2-40ba-abac-d05b7856c5e6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.405120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d17f-2dd2-433b-b4e7-d94990b9e912 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.444786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.445707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.446132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.446548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.447472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:42.447919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:42.449485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004113-677d-4ae7-b8be-146a9a8f3311 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:42.458648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 33/68] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:42.460162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.460603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.461036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.479607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:42.481063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:42.502015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa705e3-95b2-40ba-abac-d05b7856c5e6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.522862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:42.527317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:42.550295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4a23-9111-4fec-ac90-9e4b181f80a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.551790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4a23-9111-4fec-ac90-9e4b181f80a7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.552163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4a23-9111-4fec-ac90-9e4b181f80a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.552446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4a23-9111-4fec-ac90-9e4b181f80a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.559619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 34/69] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:42.560776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.563963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e4a23-9111-4fec-ac90-9e4b181f80a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-115e4a23-9111-4fec-ac90-9e4b181f80a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:42.573201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:42.574314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:42.575171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:42.577574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 36/70] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:42.579021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650f16-5a99-47e8-9534-d8d13f6492ab None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.579502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650f16-5a99-47e8-9534-d8d13f6492ab None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.579991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650f16-5a99-47e8-9534-d8d13f6492ab None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.581566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 37/71] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:42.583655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650f16-5a99-47e8-9534-d8d13f6492ab None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.681042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e947-3868-4dd4-a9b9-81f1bc37d3b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.703050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650f16-5a99-47e8-9534-d8d13f6492ab None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.724247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3d388-d2a3-41a8-8648-e53bfd997251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.725262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3d388-d2a3-41a8-8648-e53bfd997251 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.725552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3d388-d2a3-41a8-8648-e53bfd997251 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.725851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3d388-d2a3-41a8-8648-e53bfd997251 None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.741864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.742039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3d388-d2a3-41a8-8648-e53bfd997251 None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:42.742538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.742826 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.743102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.743428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3d388-d2a3-41a8-8648-e53bfd997251 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:42.743854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:42.744163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:42.745184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 35/72] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:42.745649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:42.746372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1065ae-a74d-4161-8a0a-09993b4429ed None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.746665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1065ae-a74d-4161-8a0a-09993b4429ed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.746941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1065ae-a74d-4161-8a0a-09993b4429ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.757179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 38/73] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:42.757927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.764042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5e1065ae-a74d-4161-8a0a-09993b4429ed None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:42.765266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5e1065ae-a74d-4161-8a0a-09993b4429ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:42.790256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:42.806041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:42.838482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 36/74] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:42.839821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1601-99b0-4be2-b2de-6d6c07366c53 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.840122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1601-99b0-4be2-b2de-6d6c07366c53 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.840410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e1601-99b0-4be2-b2de-6d6c07366c53 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.842301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 37/75] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:42.847363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1601-99b0-4be2-b2de-6d6c07366c53 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.867136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc9d5a-5176-40d4-a11a-c47f38786989 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.900847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.901674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.901962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.902240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:42.903671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:42.905155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:42.906772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 39/76] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:42.907538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:42.954363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e1601-99b0-4be2-b2de-6d6c07366c53 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:42.988105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:42.988748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:42.989072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:42.989352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.009425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cae06-47e6-4737-ac81-b0b8c406763d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.016952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.018024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.045279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87e93e-d352-461d-85c4-f2707ef1690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.046163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87e93e-d352-461d-85c4-f2707ef1690e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.046450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87e93e-d352-461d-85c4-f2707ef1690e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.046728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87e93e-d352-461d-85c4-f2707ef1690e None admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.047926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87e93e-d352-461d-85c4-f2707ef1690e None admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.049419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87e93e-d352-461d-85c4-f2707ef1690e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.051691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 40/77] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:43.052834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.053217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.053500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.063979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.065507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.070058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:43.071240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:43.136444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 41/78] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:43.149763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.166642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 38/79] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:43.171569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.226401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.252813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a0510-c220-4c34-9386-679e85d703ca None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.261804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.268947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418744-ffdb-4337-a83c-cc402e1ca16d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.290405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.291224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.291516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.291796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.302855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:43.303241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.303904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.304185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.304461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.305206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.306644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:43.311382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:43.312232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:43.313037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 42/80] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:43.313669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.331680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 39/81] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:13:43 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:43.333103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.450553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79146dc-5e69-4320-831b-b180efe19092 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.456781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846914ca-9b95-4148-8804-3e28833869fb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.475498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.492940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.493619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.493910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.494200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.499865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.500677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.501001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.501285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.515898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:43.524723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:43.525816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:43.526602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 40/82] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:43.527222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.533715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.573016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.574535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.669901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19db18-db16-4df2-b4a0-2aeab00532fa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.716099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 43/83] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:43.717481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.729458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.730747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.731241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.731745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.733272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.734993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.736964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 41/84] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:43.737941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.788389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.803405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.844682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92200f1d-f31d-4457-ab0a-ab6b4135ff9b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.863614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17448863-43a1-4561-bb45-9a9e1fe749df None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.885558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.887096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.887623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.888104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.889181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:43.890230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.892373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.906341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 44/85] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:43.911013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.911013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.911013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.911013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.911327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:43.911327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.911327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.913024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.921438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 42/86] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:43.926135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.030282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a073f-e6f9-4eb5-9b52-b9cc7b0b42b8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.040300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298383-e68b-4ab4-939d-905930f0cc5a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.050615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.061741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.070841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.071707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.072106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.072316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.073547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.075338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.076966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 45/87] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:44.077848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.088005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.089145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.090021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.090767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.092553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=627b0bc28165465490f31ac21975b10e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:44.093227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.095151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.109105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 43/88] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/role_assignments?user.id=627b0bc28165465490f31ac21975b10e&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:44.110499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.220350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545d337-c748-47ef-b3e2-764acde0224b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.244992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a7de-b884-4a29-a974-b6c483128bf5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.285059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.286630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.286905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.287353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.287802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.288036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.288157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.288373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.289946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.291708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.293730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.293986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 44/89] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:44.294971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.295221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.296971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 46/90] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:44.303095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.342577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.411378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a331d4-905a-4b5d-8b80-f2b355e7a5ea None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.420700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.476905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.477579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.477874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.478160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.479019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=0b2e9296115548cda14048313e8cf43e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:44.479337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.480800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.493292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf03feb-fd61-4d6f-8126-b902875f8ec6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.500737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 45/91] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/role_assignments?user.id=0b2e9296115548cda14048313e8cf43e&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:44.501908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:44.574439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.575658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.575948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.576229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.713918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.715475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.734531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc924b5-7612-45c1-afeb-2dd0ad3bf566 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:44.737548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.805885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:44.806574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:44.807048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:44.807339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:44.874026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:44.936046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:44.937552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:44.952518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 47/92] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:13:44.959911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.125054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64223ceb-0197-4531-a372-3bcf2f2f7948 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.131752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 46/93] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:45.140709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.142953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.202332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.203370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.203657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.203933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.206488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.208739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.210848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 48/94] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:44 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:45.222633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.275354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.403517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bf2e5-1946-4ce8-b2be-3477e805f59b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.419456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c182d5-f886-43df-8c0c-599de680d6d5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.491564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.492231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.492514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.492791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.493661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] RBAC: Adding query filter params (role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=627b0bc28165465490f31ac21975b10e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:45.493977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.495421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd47770-0839-4162-b999-a556224ac966 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.497593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.498465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.498753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.499032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] PATH_INFO: `/v3/projects/0effce8b88124b3f8d1fdf7c1652440b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.500299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] RBAC: Authorizing `identity:get_project(project_id=0effce8b88124b3f8d1fdf7c1652440b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.501790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.503528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 47/95] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/projects/0effce8b88124b3f8d1fdf7c1652440b => generated 312 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:45.507805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 49/96] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/role_assignments?user.id=627b0bc28165465490f31ac21975b10e&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:45.510138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:45.619130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.664619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e753dc-25de-44c6-81d0-efd6bc2aa3da None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:45.718325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:45.719002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.719293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.719578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.720436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=0effce8b88124b3f8d1fdf7c1652440b, user.id=627b0bc28165465490f31ac21975b10e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:45.720750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:45.722247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56aac8-695f-4c35-8649-54c8b78684f9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:45.733597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 48/97] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:45 2022] GET /identity/v3/role_assignments?user.id=627b0bc28165465490f31ac21975b10e&scope.project.id=0effce8b88124b3f8d1fdf7c1652440b&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:45.885116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442444ae-b89c-4826-9f65-8c20d0c5275e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.885754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442444ae-b89c-4826-9f65-8c20d0c5275e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.886227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442444ae-b89c-4826-9f65-8c20d0c5275e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.887993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 50/98] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:45.893924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199a7c1-55fe-4626-ab1c-b3ff4084319d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.894222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199a7c1-55fe-4626-ab1c-b3ff4084319d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.895040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199a7c1-55fe-4626-ab1c-b3ff4084319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.915278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b199a7c1-55fe-4626-ab1c-b3ff4084319d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:45.917003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b199a7c1-55fe-4626-ab1c-b3ff4084319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:45.995105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 49/99] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:46.021550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442444ae-b89c-4826-9f65-8c20d0c5275e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.105987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6840d7-4522-455a-97c4-dd2e0dc989ec None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.106615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6840d7-4522-455a-97c4-dd2e0dc989ec None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.107091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6840d7-4522-455a-97c4-dd2e0dc989ec None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.108757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 50/100] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:46.121537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.122502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.123005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.144223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:46.147164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:46.148486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.165647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442444ae-b89c-4826-9f65-8c20d0c5275e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.224324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.236259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 51/101] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:46.255065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.255905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.256198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.256481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.267027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.281937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:46.291006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:46.291870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:13:46.292636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:46.297347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 51/102] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:46.319306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.397337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5545a-e19b-4485-92ff-50c13892c9d4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.445906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.447078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.447559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.448045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.449797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.451554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.453419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 52/103] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:46.459663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.558268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21488cb9-ca6f-4c20-8be7-f6cbc868c682 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.619717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.631642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.632324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.632833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.633160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.633931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:46.634249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.635658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.655643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 52/104] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:46.662621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.689953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411ad0-fb65-4a84-af28-17c15f8eaf96 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.704926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.760457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.761505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.761980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.762473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.763384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.765052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.813560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 53/105] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:13:46 2022] POST /identity/v3/projects => generated 302 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:13:46.827340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525db08-9f59-4c01-8323-4c7036d54fd8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.881968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c216e-c495-4794-8b9b-01446eecfd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.882888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c216e-c495-4794-8b9b-01446eecfd1e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.883174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c216e-c495-4794-8b9b-01446eecfd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.883489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c216e-c495-4794-8b9b-01446eecfd1e None admin] PATH_INFO: `/v3/users/937ba9a7e0e444a5bc4b2a5716566d4c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.903686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c216e-c495-4794-8b9b-01446eecfd1e None admin] RBAC: Authorizing `identity:get_user(user_id=937ba9a7e0e444a5bc4b2a5716566d4c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.905578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c216e-c495-4794-8b9b-01446eecfd1e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.907554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 53/106] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/users/937ba9a7e0e444a5bc4b2a5716566d4c => generated 289 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:46.914260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:47.070446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f9310-f96e-4819-b21b-3d9077de4e60 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:47.086528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:47.110365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:47.111027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.111314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.111590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.170951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de9b2-3443-43e6-a3d4-46016c3c236c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.171965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de9b2-3443-43e6-a3d4-46016c3c236c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.172474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918de9b2-3443-43e6-a3d4-46016c3c236c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.174428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 54/107] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:47.182211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.182742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.183243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.188908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:47.189396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:47.233924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:47.235471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:47.295335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 54/108] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:47.301509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:47.319447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:47.410221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab3ab-117c-46e9-aa0b-3c2ab38a0686 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:47.438457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 55/109] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:47.459752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:47.460735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.461068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.461350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.463161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:47.463725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:47.464614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:47.466816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 55/110] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:47 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:47.473862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:47.479203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:47.624955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:47.643423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfccdf-d737-4e57-b600-52779be5e22b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:47.674140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87613f78-86d4-46fd-8bd4-6f751afe4f55 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:47.680744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:47.691601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.691917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.692182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.693090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] RBAC: Adding query filter params (role.id=41a64573139b4495837c399b914a63b3, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=937ba9a7e0e444a5bc4b2a5716566d4c) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:47.693415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:47.703632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7cd1-9d22-45e1-acdf-f93ef94ff486 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:47.709883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 56/111] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:47 2022] GET /identity/v3/role_assignments?user.id=937ba9a7e0e444a5bc4b2a5716566d4c&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=41a64573139b4495837c399b914a63b3 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:47.726777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592315ca-294b-406c-8ce5-cd790f00edcc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.727087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592315ca-294b-406c-8ce5-cd790f00edcc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.727358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592315ca-294b-406c-8ce5-cd790f00edcc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.729004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 57/112] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:47.734409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.734706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.734991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.761648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:47.762475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:47.762766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:47.763050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:47.764699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:47.765968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:47.789095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:47.797124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:47.798007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:47.798778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 56/113] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:47.807441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:47.887629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 58/114] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:47.911698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:47.976989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:47.999807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.028611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c035c-a711-4798-a427-85e02c3e1df7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.072758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fba58-d425-4c03-9bb5-6da9685cafc7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.091779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.092598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.092903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.093311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.117399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:48.129891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:48.130935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:48.131696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 59/115] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:48.137561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.147696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.148391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.148681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.149056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.149779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:48.150087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.151512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.163087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 57/116] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:48.169072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.380009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0559b6-55ef-4762-97e7-0e8b1fa6b4f0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.417334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f5995-aeab-45e0-b111-2a94ef50f8f6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.443466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.460304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.461461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.462543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.462982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.465945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.468260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.470339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 58/117] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:48.476712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.492905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.492905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.492905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.492905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.493292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:48.493292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.493914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.499956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.511326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 60/118] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:48.517020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.649924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bf24f-a215-4259-94e9-547b78577879 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.706539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.707955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.708246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.708531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.710129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.711241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.713000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 59/119] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:48.722019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.752766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1551e54-2c12-4233-9b6d-30c7c1d8d13e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.820702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c97d-275a-4249-9478-385f61eb4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.821775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c97d-275a-4249-9478-385f61eb4d46 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.822067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c97d-275a-4249-9478-385f61eb4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.822353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c97d-275a-4249-9478-385f61eb4d46 None admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.824134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938c97d-275a-4249-9478-385f61eb4d46 None admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.825766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938c97d-275a-4249-9478-385f61eb4d46 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.827439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 61/120] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:48.828703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba7a12-5a95-49ee-800d-19067ed59e43 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.829027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba7a12-5a95-49ee-800d-19067ed59e43 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.829373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba7a12-5a95-49ee-800d-19067ed59e43 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.831162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 62/121] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:48.842125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba7a12-5a95-49ee-800d-19067ed59e43 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.876195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.945011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.965348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51e0ff-35f5-4eae-b587-69e9cc2304d0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.990530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba7a12-5a95-49ee-800d-19067ed59e43 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.002160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ebe2-33cc-493d-b6fe-3e010de257e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.003065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ebe2-33cc-493d-b6fe-3e010de257e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.003357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ebe2-33cc-493d-b6fe-3e010de257e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.004433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ebe2-33cc-493d-b6fe-3e010de257e8 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b/users/627b0bc28165465490f31ac21975b10e/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.006608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4ebe2-33cc-493d-b6fe-3e010de257e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f00ea992c824af582c7fbffe696728b, user_id=627b0bc28165465490f31ac21975b10e, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.008103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4ebe2-33cc-493d-b6fe-3e010de257e8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.027135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 60/122] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:48 2022] PUT /identity/v3/projects/3f00ea992c824af582c7fbffe696728b/users/627b0bc28165465490f31ac21975b10e/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:49.033590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317204f7-7db2-4dc9-8233-fe422673dbfd None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.034343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317204f7-7db2-4dc9-8233-fe422673dbfd None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.034827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317204f7-7db2-4dc9-8233-fe422673dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.042339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed6064-58d3-426f-a4c7-b77891242feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.043649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed6064-58d3-426f-a4c7-b77891242feb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.044140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed6064-58d3-426f-a4c7-b77891242feb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.044648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed6064-58d3-426f-a4c7-b77891242feb None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.046941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed6064-58d3-426f-a4c7-b77891242feb None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.049806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed6064-58d3-426f-a4c7-b77891242feb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.062065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 63/123] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:49.063956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6c1b9-9a91-4b74-a936-7bb722d05f6e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.067326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6c1b9-9a91-4b74-a936-7bb722d05f6e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.067621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6c1b9-9a91-4b74-a936-7bb722d05f6e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.069183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 64/124] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:49.070676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6c1b9-9a91-4b74-a936-7bb722d05f6e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.083575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-317204f7-7db2-4dc9-8233-fe422673dbfd None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:49.084822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-317204f7-7db2-4dc9-8233-fe422673dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:49.225912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 61/125] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:49.227341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.227641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.227931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.256581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:49.258008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:49.258329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6c1b9-9a91-4b74-a936-7bb722d05f6e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.295669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.296617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.296977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.297264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b/users/0b2e9296115548cda14048313e8cf43e/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.299336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] RBAC: Authorizing `identity:create_grant(project_id=3f00ea992c824af582c7fbffe696728b, user_id=0b2e9296115548cda14048313e8cf43e, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.301532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.306514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.323049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 65/126] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:49 2022] PUT /identity/v3/projects/3f00ea992c824af582c7fbffe696728b/users/0b2e9296115548cda14048313e8cf43e/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:49.325101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.360070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.396265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 62/127] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:49.433662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.484105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6585f-2bbd-4948-8a62-d85ee9dccd5a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.545770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.546605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.546895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.547176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.571037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:49.579096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:49.580003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:49.580777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 66/128] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:49.589195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.666931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed335c-1327-4ee8-8798-f40bc6345818 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.677258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.725633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.726492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.726785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.727070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.742496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:49.748546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c730-fbce-4c7a-8552-b8661a2f34a2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:49.755491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:49.756398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:49.757200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 63/129] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:49.765833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.813611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:49.814297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:49.814590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:49.814874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:49.815638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:49.815948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:49.817918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:49.835020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 67/130] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:49.840910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:49.843302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:49.940224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e674-ac48-4073-91fb-833c093d75be None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:49.997077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b3a72-2ed4-428d-a697-aac167814360 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.005694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.006397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.006742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.007038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.007870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:50.008200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.009802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.026913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 64/131] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.033049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.041272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.062180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.063073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.063362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.063643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.070347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.071824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.073827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 68/132] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:49 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.079282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.164042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e01047-1e81-4669-a3ff-01f4bcbe14c9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.172549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.221426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d466c8-3b31-4741-aa3b-086c2498a506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.222312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d466c8-3b31-4741-aa3b-086c2498a506 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.222636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d466c8-3b31-4741-aa3b-086c2498a506 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.222976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d466c8-3b31-4741-aa3b-086c2498a506 None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.224430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d466c8-3b31-4741-aa3b-086c2498a506 None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.227428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d466c8-3b31-4741-aa3b-086c2498a506 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.229310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 65/133] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:50.230455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e7ada-462b-4075-9544-36a52d237ae5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.230627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e7ada-462b-4075-9544-36a52d237ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.230908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e7ada-462b-4075-9544-36a52d237ae5 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.232138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 66/134] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:50.236461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e7ada-462b-4075-9544-36a52d237ae5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.262946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282d347-c9d5-41e3-9e0c-8d4226033bad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.321411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f58251-e43e-461e-b27f-4f4515cd924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.322286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f58251-e43e-461e-b27f-4f4515cd924d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.322567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f58251-e43e-461e-b27f-4f4515cd924d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.322843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f58251-e43e-461e-b27f-4f4515cd924d None admin] PATH_INFO: `/v3/projects/0effce8b88124b3f8d1fdf7c1652440b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.324169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f58251-e43e-461e-b27f-4f4515cd924d None admin] RBAC: Authorizing `identity:get_project(project_id=0effce8b88124b3f8d1fdf7c1652440b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.328082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f58251-e43e-461e-b27f-4f4515cd924d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.330021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 69/135] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/projects/0effce8b88124b3f8d1fdf7c1652440b => generated 312 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:50.331145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.331317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.331587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.370200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:50.371853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:50.396371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e7ada-462b-4075-9544-36a52d237ae5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.462301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.463227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.463520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.463799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.466080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.471142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.473295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 67/136] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.474193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.475993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.531091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 70/137] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:50.531677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.531749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.592405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fb487-2c75-4550-862b-f2d33dd3da63 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.641677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.644135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.644979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.645714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] PATH_INFO: `/v3/projects/0effce8b88124b3f8d1fdf7c1652440b/users/627b0bc28165465490f31ac21975b10e/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.650444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] RBAC: Authorizing `identity:create_grant(project_id=0effce8b88124b3f8d1fdf7c1652440b, user_id=627b0bc28165465490f31ac21975b10e, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.653988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.683599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 68/138] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:50 2022] PUT /identity/v3/projects/0effce8b88124b3f8d1fdf7c1652440b/users/627b0bc28165465490f31ac21975b10e/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:50.684091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:50.720507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db9c74-35c0-4d71-b01a-f9322eb69c49 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.760364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.761710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.762201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.762677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b/users/627b0bc28165465490f31ac21975b10e/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.765176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f00ea992c824af582c7fbffe696728b, user_id=627b0bc28165465490f31ac21975b10e, role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.766883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.785595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 71/139] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:50 2022] PUT /identity/v3/projects/3f00ea992c824af582c7fbffe696728b/users/627b0bc28165465490f31ac21975b10e/roles/a0ef0911aac646be8916503dba5a9c3f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:50.888104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb5455-8daa-40c1-8516-6c98cccd4b26 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:50.890302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:50.935099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b454f-d88b-45f1-a780-91bf5d6d5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:50.935929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b454f-d88b-45f1-a780-91bf5d6d5081 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.936222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b454f-d88b-45f1-a780-91bf5d6d5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.936505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b454f-d88b-45f1-a780-91bf5d6d5081 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.937836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b454f-d88b-45f1-a780-91bf5d6d5081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:50.939356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b454f-d88b-45f1-a780-91bf5d6d5081 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:50.941918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 69/140] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:50.947881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.000261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:51.001700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdae1fa-7534-4fbb-8c2b-0af23a501fc8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.002540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdae1fa-7534-4fbb-8c2b-0af23a501fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.002540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdae1fa-7534-4fbb-8c2b-0af23a501fc8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.003833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 70/141] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:51.014413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.014736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.014967 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.043792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:51.045098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:51.113887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4fa41-757f-4170-8193-842fab590ef0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.170812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 71/142] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:51.173896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.174586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.174883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.175136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.176709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.178217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.207981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 72/143] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:13:50 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:51.212521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.356660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:51.361298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda237-0b1d-4162-8653-24a396d84af8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.406398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.407226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.407515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.407792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.420104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:51.428418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:51.429415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:51.430206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 72/144] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:51.437315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.565481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:51.575101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f492-2029-4b1b-9385-2da5008f6144 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.623084 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.623694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.623983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.624265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.625061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:51.625376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.626870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.640680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 73/145] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:51.647121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.782170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23698c88-cae5-487c-9880-aa341a323451 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.827055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.827955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.828245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.828523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] PATH_INFO: `/v3/users/937ba9a7e0e444a5bc4b2a5716566d4c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.829817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] RBAC: Authorizing `identity:get_user(user_id=937ba9a7e0e444a5bc4b2a5716566d4c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.831278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.834592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 73/146] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:51 2022] GET /identity/v3/users/937ba9a7e0e444a5bc4b2a5716566d4c => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:51.839367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.974410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06983a37-2efe-4b88-a70a-4740bc3e9d93 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:52.013055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e8843-d7c3-4623-ad14-5ec195407d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:52.013982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8843-d7c3-4623-ad14-5ec195407d64 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.014310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8843-d7c3-4623-ad14-5ec195407d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.014594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e8843-d7c3-4623-ad14-5ec195407d64 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.017713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e8843-d7c3-4623-ad14-5ec195407d64 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:52.019281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e8843-d7c3-4623-ad14-5ec195407d64 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:52.021596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 74/147] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:51 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:52.028527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:52.079890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:52.162965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e14bd-4aa1-404b-b10a-510059e48d2c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:52.202741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65329beb-a025-4736-8fe7-597e69942c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:52.203678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65329beb-a025-4736-8fe7-597e69942c14 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.203973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65329beb-a025-4736-8fe7-597e69942c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.204256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65329beb-a025-4736-8fe7-597e69942c14 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b/users/937ba9a7e0e444a5bc4b2a5716566d4c/roles/41a64573139b4495837c399b914a63b3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.210221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65329beb-a025-4736-8fe7-597e69942c14 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f00ea992c824af582c7fbffe696728b, user_id=937ba9a7e0e444a5bc4b2a5716566d4c, role_id=41a64573139b4495837c399b914a63b3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:52.211713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65329beb-a025-4736-8fe7-597e69942c14 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:52.231456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 74/148] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:52 2022] PUT /identity/v3/projects/3f00ea992c824af582c7fbffe696728b/users/937ba9a7e0e444a5bc4b2a5716566d4c/roles/41a64573139b4495837c399b914a63b3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:52.270635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9560970-84e2-4e04-84b7-e806bf0f6d94 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.271125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9560970-84e2-4e04-84b7-e806bf0f6d94 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.271420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9560970-84e2-4e04-84b7-e806bf0f6d94 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.272843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 75/149] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:52.279357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.279665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.279934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.302645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:52.303888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:52.329000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0942-0248-41bc-88ac-e9e90e1dfb57 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.329323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0942-0248-41bc-88ac-e9e90e1dfb57 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.329584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0942-0248-41bc-88ac-e9e90e1dfb57 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.333062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 76/150] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:52.340819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.341281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.341454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.372203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:52.373494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:52.394734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 75/151] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:52.434043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:52.435240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:52.484452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:52.517198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 77/152] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:52.583626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:52.595659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d74e6-cbb2-4417-89b5-7e369ce6a485 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:52.658695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:52.659515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.659842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.660725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.682310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:52.705723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:52.706672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:52.708070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 76/153] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:52.716501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:52.793769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a8ea-79ef-4f75-abe1-4580e9a4947a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:52.861274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:52.862091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.862383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.862661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.865561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:52.885500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:52.895281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:52.896266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:52.897166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:52.901567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 78/154] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:52.911373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee5829-6bb1-4b27-bfd8-6e7862375187 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:52.929572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:52.961192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:52.983166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:52.983939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:52.984229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:52.984510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:52.985324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:52.985636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:52.987052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.003114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 77/155] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:53.009577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.103828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab6523-fd45-460a-a068-dfab8ca4ebc4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.169729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.170793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.171298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.171792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.172801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:53.173337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.174944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.196255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 79/156] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:53.206570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.249437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358cd9b-af9c-489f-b82d-b8d3106ad0d3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.309825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.310723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.311017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.311293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.313453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.315046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.318031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 78/157] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:53.323487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.348849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.378595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.417024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80de4c7-e95f-41f9-a29e-0eb7603a227c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.491886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.493175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.493666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.494173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] PATH_INFO: `/v3/users/0b2e9296115548cda14048313e8cf43e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.495648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] RBAC: Authorizing `identity:get_user(user_id=0b2e9296115548cda14048313e8cf43e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.497374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.499547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 80/158] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/users/0b2e9296115548cda14048313e8cf43e => generated 247 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:53.509724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.525098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8807db-619f-45e1-9a82-0f3f144a8d0f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.587801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.588489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.588754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.589067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.683158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1403a-8cb4-406e-b514-f128c837a9be None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:53.712538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.714160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:53.728848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.774563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:53.775583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:53.776071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:53.776570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:53.810658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:53.863334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 79/159] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:53.869115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:53.938128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:53.940043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.027287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60128e13-dc2e-4917-b96b-80a3aa5c585e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.046341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.079546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.080531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.080823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.081125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.085655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.087278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.089217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 80/160] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:54.095451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.188789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 81/161] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:13:54.195123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.297573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5c0a-7ad0-4317-82b9-888dd2260721 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.301706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.384020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df5281-a01c-4234-a698-2425e1e3afd9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.432089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ce097-9079-456f-a51b-a25926529188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.432982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ce097-9079-456f-a51b-a25926529188 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.433276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ce097-9079-456f-a51b-a25926529188 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.433556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ce097-9079-456f-a51b-a25926529188 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.437604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ce097-9079-456f-a51b-a25926529188 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.439140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ce097-9079-456f-a51b-a25926529188 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.441528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 82/162] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:54.447460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ce097-9079-456f-a51b-a25926529188 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.449505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.450190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.450470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.450742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.451599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=627b0bc28165465490f31ac21975b10e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:54.451913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.453413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c0a57-128c-4643-92a9-b41771f7ad1a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.466850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 81/163] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/role_assignments?user.id=627b0bc28165465490f31ac21975b10e&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:54.587709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ce097-9079-456f-a51b-a25926529188 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.611152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2d4c7-82bb-4ceb-a935-8a08d0ae5d7f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.611656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2d4c7-82bb-4ceb-a935-8a08d0ae5d7f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.611761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2d4c7-82bb-4ceb-a935-8a08d0ae5d7f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.613178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 82/164] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:54.619079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.619354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.619631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.625298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.636655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:54.637949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:54.642023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.643222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.643515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.643800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.645379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=0b2e9296115548cda14048313e8cf43e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:54.645701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:54.647156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:54.653766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 83/165] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/role_assignments?user.id=0b2e9296115548cda14048313e8cf43e&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:54.729217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:54.775864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 83/166] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:54.799661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:54.928284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806c62e-1d68-4af0-b12d-74c6dcef11df None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:54.966122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:54.967010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:54.967297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:54.967570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:54.980754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:54.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:54.990570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:54.992020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 84/167] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:54.998001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.130419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc88a6-b008-4b73-b1eb-a1909fa822c5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.171723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.172405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.172690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.173009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.173776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:55.174085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.175530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.184615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 84/168] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:55.190384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.237006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:55.304382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f819-dd78-4753-bce9-eb81494041b1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.341165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619fd04-3561-4adb-83df-109122a161e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.342050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619fd04-3561-4adb-83df-109122a161e3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.342334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619fd04-3561-4adb-83df-109122a161e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.342606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619fd04-3561-4adb-83df-109122a161e3 None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.346110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619fd04-3561-4adb-83df-109122a161e3 None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.348577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619fd04-3561-4adb-83df-109122a161e3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.350351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 85/169] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:55.356006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.381021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b958a70-aded-475f-80fa-077c5c5bd247 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.381306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b958a70-aded-475f-80fa-077c5c5bd247 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.381587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b958a70-aded-475f-80fa-077c5c5bd247 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.383258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 86/170] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:55.394972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.395266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.395548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.420247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:55.421540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:55.426626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:55.528998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebde2f5-020a-4e08-834c-c9f5ac260a0f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.568702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 87/171] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:13:55.574360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.575025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.575314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.575591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.601695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.663843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:55.665845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:55.700985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:55.728587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:55.865780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494f0d1-05a8-4a71-930a-1a4bc2b164e7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:55.866302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 85/172] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:55.871728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:55.954263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21777a-1331-4d68-8617-a9f06f5d19a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:55.955104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21777a-1331-4d68-8617-a9f06f5d19a9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:55.955391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21777a-1331-4d68-8617-a9f06f5d19a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:55.955669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21777a-1331-4d68-8617-a9f06f5d19a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:55.985817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21777a-1331-4d68-8617-a9f06f5d19a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1a21777a-1331-4d68-8617-a9f06f5d19a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:55.995633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:55.996846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:55.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:55.999815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 88/173] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:56.001291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aba78-7ba6-4afa-9c1d-d7029af65252 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.001728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aba78-7ba6-4afa-9c1d-d7029af65252 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.002148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aba78-7ba6-4afa-9c1d-d7029af65252 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.008164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 89/174] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:56.009267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aba78-7ba6-4afa-9c1d-d7029af65252 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.021424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e09844-36ac-435e-9809-7b307ff3b5c1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.057442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:56.067005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d283ea2-1bfe-42c9-8e5c-8e6f39d0a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.067900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d283ea2-1bfe-42c9-8e5c-8e6f39d0a110 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.068186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d283ea2-1bfe-42c9-8e5c-8e6f39d0a110 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.068474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d283ea2-1bfe-42c9-8e5c-8e6f39d0a110 None admin] PATH_INFO: `/v3/projects/0effce8b88124b3f8d1fdf7c1652440b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.072572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d283ea2-1bfe-42c9-8e5c-8e6f39d0a110 None admin] RBAC: Authorizing `identity:get_project(project_id=0effce8b88124b3f8d1fdf7c1652440b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:56.074097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d283ea2-1bfe-42c9-8e5c-8e6f39d0a110 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:56.075857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 86/175] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:55 2022] GET /identity/v3/projects/0effce8b88124b3f8d1fdf7c1652440b => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:56.078012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.078301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.078583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.113055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:56.114331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:56.198053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 87/176] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:56.204722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.243629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:56.252433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aba78-7ba6-4afa-9c1d-d7029af65252 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.327430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.328200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.328455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.328770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.329572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:56.329975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:56.331492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:56.334952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c255b-f534-47c1-8794-59147b8fce8a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.355377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 90/177] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:56.356266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.377375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:56.379489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.380789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.381255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.381678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.382709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=0effce8b88124b3f8d1fdf7c1652440b, user.id=627b0bc28165465490f31ac21975b10e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:56.383173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:56.384796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:56.392292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 88/178] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/role_assignments?user.id=627b0bc28165465490f31ac21975b10e&scope.project.id=0effce8b88124b3f8d1fdf7c1652440b&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:56.393346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.536682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6ebd1-4050-4091-be87-6243922825f6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.571502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952512a-614f-4685-b276-f9ebccdcbe47 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.590906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e01179-3e12-45ee-b8c7-b4dfd527597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.591966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01179-3e12-45ee-b8c7-b4dfd527597d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.592407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01179-3e12-45ee-b8c7-b4dfd527597d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.592862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01179-3e12-45ee-b8c7-b4dfd527597d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.612035 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e01179-3e12-45ee-b8c7-b4dfd527597d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-42e01179-3e12-45ee-b8c7-b4dfd527597d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:56.620318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:56.621308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:13:56.622250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 91/179] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:56.623434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce6fcc-a01d-4865-b822-a2d85fa2d041 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.623434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce6fcc-a01d-4865-b822-a2d85fa2d041 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.623554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce6fcc-a01d-4865-b822-a2d85fa2d041 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.623873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a31ed-f0f0-4651-a5a6-99f2b4d36b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.625150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a31ed-f0f0-4651-a5a6-99f2b4d36b7d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.625326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 92/180] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:56.625984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d691c8-cda7-4c99-80c8-2950a0a2f553 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.626300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d691c8-cda7-4c99-80c8-2950a0a2f553 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.626531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d691c8-cda7-4c99-80c8-2950a0a2f553 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.627900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 93/181] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:56.628994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d691c8-cda7-4c99-80c8-2950a0a2f553 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.629823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a31ed-f0f0-4651-a5a6-99f2b4d36b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.630405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a31ed-f0f0-4651-a5a6-99f2b4d36b7d None admin] PATH_INFO: `/v3/users/627b0bc28165465490f31ac21975b10e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.631999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a31ed-f0f0-4651-a5a6-99f2b4d36b7d None admin] RBAC: Authorizing `identity:get_user(user_id=627b0bc28165465490f31ac21975b10e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:56.634850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a31ed-f0f0-4651-a5a6-99f2b4d36b7d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:56.638367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 89/182] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/users/627b0bc28165465490f31ac21975b10e => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:56.639821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f55ce-b5ff-4cec-a576-3f7a969ec208 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.640244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f55ce-b5ff-4cec-a576-3f7a969ec208 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.640653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f55ce-b5ff-4cec-a576-3f7a969ec208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.663358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:56.665344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fb7f55ce-b5ff-4cec-a576-3f7a969ec208 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:56.666694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fb7f55ce-b5ff-4cec-a576-3f7a969ec208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:56.739320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:56.751470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 90/183] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:56.752717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dce6e-24a2-45c2-bbb4-8a21a7b7e6b4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.753048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dce6e-24a2-45c2-bbb4-8a21a7b7e6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.753333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dce6e-24a2-45c2-bbb4-8a21a7b7e6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.766405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d691c8-cda7-4c99-80c8-2950a0a2f553 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.778931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2b1dce6e-24a2-45c2-bbb4-8a21a7b7e6b4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:56.780145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2b1dce6e-24a2-45c2-bbb4-8a21a7b7e6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:56.802213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.802819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.803103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.803380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.804127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:56.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:56.805893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:56.814488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 94/184] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:56.818156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.852333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 91/185] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:56.853663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f64a3-f552-48da-8d53-1a56bc54637f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.853951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f64a3-f552-48da-8d53-1a56bc54637f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.854225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f64a3-f552-48da-8d53-1a56bc54637f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.855491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 92/186] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:56.856527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f64a3-f552-48da-8d53-1a56bc54637f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:56.907518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:56.929577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315fbb3-ba7a-4efd-8733-d990a96d9db6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.965279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08034b9-565f-46c7-8eae-f68c637fa180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:56.965921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08034b9-565f-46c7-8eae-f68c637fa180 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:56.966200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08034b9-565f-46c7-8eae-f68c637fa180 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:56.966473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08034b9-565f-46c7-8eae-f68c637fa180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:56.969357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f64a3-f552-48da-8d53-1a56bc54637f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:56.992556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.005332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.006129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.006407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.006679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.017227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:57.025948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:57.026951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:13:57.027773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:57.029411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 93/187] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:57.030245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.036582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08034b9-565f-46c7-8eae-f68c637fa180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08034b9-565f-46c7-8eae-f68c637fa180 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.119641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.134706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00feb0-323c-4e61-8967-5248f27c1d5b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.140407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 95/188] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:57.142799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.143090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.143368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.159612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:57.161012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:57.168408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.169317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.169605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.169890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] PATH_INFO: `/v3/users/935bae62b9f0453fb193f7611f39896d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.183890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] RBAC: Authorizing `identity:get_user(user_id=935bae62b9f0453fb193f7611f39896d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.185394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.187086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 94/189] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/users/935bae62b9f0453fb193f7611f39896d => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:57.187933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.228203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 96/190] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:13:57.229655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.229954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.291782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b3bde-6b44-49b0-9c82-b9c0789de336 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.325942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.326739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.327020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.327291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.328501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.330049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.331138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45b12b-9464-4236-81e1-637412eceaad None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.336347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 95/191] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:57.336347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.351277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.375341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.376055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.376407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.376817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.377768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:57.378205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.380692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.389695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 97/192] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:57.390182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.439669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea136944-7181-4fdb-a4d5-f1f7140ec35d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.450937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.479212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.480077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.480364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.480639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.481862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.483332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.485085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 96/193] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:57.485845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.495029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f641-e382-4cac-a926-fe97b34e15a3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.544167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.545227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.545678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.546110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.598899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.610027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d7e0b-f5fa-4b93-84ea-1ac636db738e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.626297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.627720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.645721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.646522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.646805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.647082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.657808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:57.665796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:57.667253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:13:57.668062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 97/194] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:57.668735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.710971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.737699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 98/195] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:57.739471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.781533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175597c-e312-4ea5-88be-9391f17eb83e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.814092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.814551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.815223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.815508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.815837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.817382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.818857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.839254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 98/196] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:13:57 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:57.839254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.853337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96bcd3-d587-415c-bf70-86ddbeb5d839 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:57.882039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:57.883692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:57.883977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:57.884260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] PATH_INFO: `/v3/users/935bae62b9f0453fb193f7611f39896d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:57.885700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=935bae62b9f0453fb193f7611f39896d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:57.887173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:57.894986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 99/197] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/users/935bae62b9f0453fb193f7611f39896d => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:57.894986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:57.949647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:57.963248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b9314-1126-43ea-8cf5-914383cf5ae1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.006994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.007294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.007691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.007975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.008849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] RBAC: Adding query filter params (role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=627b0bc28165465490f31ac21975b10e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:58.009194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.010683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.018937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 99/198] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/role_assignments?user.id=627b0bc28165465490f31ac21975b10e&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.019815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.023251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c0a91-aea1-4752-bc2b-26d646ce6cd2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.072341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.073353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.073822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.074349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.075457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:58.075960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.077576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.090446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 100/199] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:13:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.091330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.096490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.167700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469fb3-6e40-463a-9f31-3d3a4b45fc92 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.209367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.210353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.210645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.210925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.212289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.213787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.215761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 100/200] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.216632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.236980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801aa62-4fa5-4903-95c5-3c8767e0a4bc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.246611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.288179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.289352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.289941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.290436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.346879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49648c-7364-459e-a199-f3079dddb122 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.377946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.380571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.384605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.400602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.401533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.401818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.402097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] PATH_INFO: `/v3/users/937ba9a7e0e444a5bc4b2a5716566d4c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.404274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] RBAC: Authorizing `identity:get_user(user_id=937ba9a7e0e444a5bc4b2a5716566d4c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.405791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.408417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 101/201] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/users/937ba9a7e0e444a5bc4b2a5716566d4c => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.412911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.502785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 101/202] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:58.509435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.531793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.542784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf5be5-a86a-446d-900d-ae03ca4aac81 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.578690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.579736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.580213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.580699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.582504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=935bae62b9f0453fb193f7611f39896d) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:58.583091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.584745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.590603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 102/203] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/role_assignments?user.id=935bae62b9f0453fb193f7611f39896d&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.598749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.636530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f8124-3247-4ba8-95b2-44e142ca35be None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.647915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:58.709324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.709324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.709324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.709324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.774505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49f6c3-d120-45f1-a660-ab385fba470e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:58.845313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:58.846187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:58.846465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:58.846736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:58.848932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.850874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.858445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 103/204] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:58.859265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:58.860813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:58.864735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:58.914624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.054388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ec28c-7806-40f6-99a6-051d5846c75b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.056892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 102/205] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:13:59.063008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.085639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.134689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.135352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.135633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.135903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.136744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] RBAC: Adding query filter params (role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=935bae62b9f0453fb193f7611f39896d) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:59.137164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.138626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.155372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 104/206] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:58 2022] GET /identity/v3/role_assignments?user.id=935bae62b9f0453fb193f7611f39896d&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:59.281119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b28c4-c9c0-45c8-acaa-1cafb927d590 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.333498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398e520-30aa-4110-90d7-2b39a76f25f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.334425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398e520-30aa-4110-90d7-2b39a76f25f9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.334710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398e520-30aa-4110-90d7-2b39a76f25f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.334992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398e520-30aa-4110-90d7-2b39a76f25f9 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.336350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398e520-30aa-4110-90d7-2b39a76f25f9 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.337920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398e520-30aa-4110-90d7-2b39a76f25f9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.339680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 103/207] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:13:59 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:59.346113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:59.569305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:59.591296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b6db-65d5-4177-b6a0-ba619e7c410a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:59.686692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:59.687372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:59.687663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:59.687947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:59.688812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] RBAC: Adding query filter params (role.id=41a64573139b4495837c399b914a63b3, scope.project.id=3f00ea992c824af582c7fbffe696728b, user.id=937ba9a7e0e444a5bc4b2a5716566d4c) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:59.693721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:59.695646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0dfd4-bc68-4636-8232-fb3b1ccc4db0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:59.711698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 105/208] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:59 2022] GET /identity/v3/role_assignments?user.id=937ba9a7e0e444a5bc4b2a5716566d4c&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=41a64573139b4495837c399b914a63b3 => generated 608 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:00.515348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f065-da13-44ab-b277-1e4140864ea8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.516375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f065-da13-44ab-b277-1e4140864ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.516862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f065-da13-44ab-b277-1e4140864ea8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.518730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 104/209] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:00.525336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.526195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.526681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.545510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9179c-bdeb-46b7-995b-6b5783cb1d42 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.545947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:00.546348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9179c-bdeb-46b7-995b-6b5783cb1d42 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.546842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9179c-bdeb-46b7-995b-6b5783cb1d42 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.547188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:00.548629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 105/210] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:00.555094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.555669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.556166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.574153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:00.575672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:00.623416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 106/211] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:00.644750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:00.661447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 106/212] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:00.685893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:00.689309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:00.756687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:00.783271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1a006-d342-4f35-9973-e730bc2559d7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:00.808557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4cbe-28d8-424f-9af3-59adb692005e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:00.828808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:00.829804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.830094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.830374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.832320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:00.834417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:00.836818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 107/213] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:00.841970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:00.854503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:00.855731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.856215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.856707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.869060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:00.877739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:00.878795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:00.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:00.882835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 107/214] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:00.890186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:00.955103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa527ab-9070-428f-8950-587e7d168bcc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:00.961102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:00.996890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47b5a3-3f46-4207-a211-abb64b2027f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:00.997594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b5a3-3f46-4207-a211-abb64b2027f8 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:00.997873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b5a3-3f46-4207-a211-abb64b2027f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:00.998154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b5a3-3f46-4207-a211-abb64b2027f8 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:00.998944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47b5a3-3f46-4207-a211-abb64b2027f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.000321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47b5a3-3f46-4207-a211-abb64b2027f8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.008590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999fcf-50c1-4021-ac6a-6a386989145e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.022555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 108/215] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:00 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:14:01.023869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473fdc0-6870-4c71-9aa7-ac500b63fd55 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.024163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473fdc0-6870-4c71-9aa7-ac500b63fd55 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.024445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473fdc0-6870-4c71-9aa7-ac500b63fd55 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.025985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 109/216] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:01.032042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.032331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.032604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.046045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.053069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:01.054574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:01.056851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.057629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.057916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.058207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.059165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:01.059272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.060859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.071550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 108/217] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:01.077221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.139392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 110/218] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:01.164407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.195567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d8ef-1880-4b6c-9fb1-8dbcd0234edd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.227315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.248000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.248917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.250443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.250443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] PATH_INFO: `/v3/users/93d97f20e7f147119f0c2b1744055253` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.267398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] RBAC: Authorizing `identity:get_user(user_id=93d97f20e7f147119f0c2b1744055253)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.268988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.271175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 109/219] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/users/93d97f20e7f147119f0c2b1744055253 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:01.276792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.284365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4839661-e8be-4588-9bdd-c95e05a92076 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.334251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.334998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.335283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.335561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.340121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.354937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:01.363397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:01.364323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:01.365149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 111/220] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:01.370884 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.420914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3fd5e-b08c-4df9-ab1d-5286db411373 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.464352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.465697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.466588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.468056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.513252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ffa41-9326-4cac-b62b-0aab5a0dcfb1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.542607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.569318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.570000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.570502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.570801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.571570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:01.571879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.579148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.579282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.579369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.602978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 112/221] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:01.609134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.688698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 110/222] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:14:01.696559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.764474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.783641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.806625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31ee85-a944-47e6-9660-b7139ca4f882 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.838709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063cc98-36f4-4f9a-93f6-d1f4a8a1b659 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:01.843170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04983f-b8e2-4452-9a82-c88898f0bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.844038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04983f-b8e2-4452-9a82-c88898f0bc25 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.844321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04983f-b8e2-4452-9a82-c88898f0bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.844599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04983f-b8e2-4452-9a82-c88898f0bc25 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.845819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04983f-b8e2-4452-9a82-c88898f0bc25 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.847944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04983f-b8e2-4452-9a82-c88898f0bc25 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.849997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 111/223] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:01.851315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2032a7-d78c-444d-8ca3-dc4d12bf41da None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.851594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2032a7-d78c-444d-8ca3-dc4d12bf41da None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.851879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2032a7-d78c-444d-8ca3-dc4d12bf41da None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.853360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 112/224] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:01.858133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2032a7-d78c-444d-8ca3-dc4d12bf41da None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:01.887453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d7665-6406-4476-b413-d535b61e0915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:01.888837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7665-6406-4476-b413-d535b61e0915 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.889795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7665-6406-4476-b413-d535b61e0915 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.890532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7665-6406-4476-b413-d535b61e0915 None admin] PATH_INFO: `/v3/users/935bae62b9f0453fb193f7611f39896d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.892155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d7665-6406-4476-b413-d535b61e0915 None admin] RBAC: Authorizing `identity:get_user(user_id=935bae62b9f0453fb193f7611f39896d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:01.894630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d7665-6406-4476-b413-d535b61e0915 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:01.897100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 113/225] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/users/935bae62b9f0453fb193f7611f39896d => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:01.899506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:01.899982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:01.900440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:01.921197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:01.922406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:01.972519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2032a7-d78c-444d-8ca3-dc4d12bf41da None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.001203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 114/226] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:02.002128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.010771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.011720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.012308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.012822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.013972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=93d97f20e7f147119f0c2b1744055253) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:02.014468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.016130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.023165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 113/227] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:01 2022] GET /identity/v3/role_assignments?user.id=93d97f20e7f147119f0c2b1744055253&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.043657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.072120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.083277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.139029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709f0b0-c533-44b2-b47d-bfb2655999ca None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.179102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.180350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.180814 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.181299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.182748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.184486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.186620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 115/228] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.200077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.271026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7f41a-e9a3-40eb-9674-f6ed1fb6f614 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.331799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfcbc8-f1f4-4e0a-b206-99d96b1cd8ab None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.354491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.355674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.356162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.356643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.361544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.376054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.377449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.377886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.378332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/935bae62b9f0453fb193f7611f39896d/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.380600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c6b6badab543ceb85ba21c445514a4, user_id=935bae62b9f0453fb193f7611f39896d, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.381936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:02.382571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:02.392976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:02.394863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:02.395779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:02.401558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 114/229] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:02.406095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.408185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 116/230] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:02 2022] PUT /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/935bae62b9f0453fb193f7611f39896d/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:14:02.496530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:02.529248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821ca64-135f-4bef-859e-4a047d2bf38d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.569879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.571301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.571737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.572158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.573143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:02.573607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.575194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.585100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 115/231] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.594081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.725180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864055e-cea4-4b5b-bcd9-a41c0c799f4b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.765532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.766784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.767232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.767655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] PATH_INFO: `/v3/users/935bae62b9f0453fb193f7611f39896d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.769657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] RBAC: Authorizing `identity:get_user(user_id=935bae62b9f0453fb193f7611f39896d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.771387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.773413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 117/232] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/users/935bae62b9f0453fb193f7611f39896d => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.781950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:02.892225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3441a0b-5429-4065-846b-f6535ae7b5be None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:02.930236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82689357-6f74-4a14-be95-db258e948b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:02.931383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82689357-6f74-4a14-be95-db258e948b9e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:02.931812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82689357-6f74-4a14-be95-db258e948b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:02.932233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82689357-6f74-4a14-be95-db258e948b9e None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:02.933608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82689357-6f74-4a14-be95-db258e948b9e None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:02.935216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82689357-6f74-4a14-be95-db258e948b9e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:02.937230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 116/233] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:02 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:02.944823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.017455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:03.084516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b4559-6661-4565-b8a5-9ec057f48469 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:03.130427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e77d0-a6b6-49cd-851d-fcced1708439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:03.131345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e77d0-a6b6-49cd-851d-fcced1708439 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.131635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e77d0-a6b6-49cd-851d-fcced1708439 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.131921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e77d0-a6b6-49cd-851d-fcced1708439 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/935bae62b9f0453fb193f7611f39896d/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.134249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e77d0-a6b6-49cd-851d-fcced1708439 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c6b6badab543ceb85ba21c445514a4, user_id=935bae62b9f0453fb193f7611f39896d, role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:03.135725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e77d0-a6b6-49cd-851d-fcced1708439 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:03.154880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 118/234] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:02 2022] PUT /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/935bae62b9f0453fb193f7611f39896d/roles/a0ef0911aac646be8916503dba5a9c3f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:03.720916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164cb4-811c-4690-8a89-0fb9eb879430 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.721455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164cb4-811c-4690-8a89-0fb9eb879430 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.721455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a164cb4-811c-4690-8a89-0fb9eb879430 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.723187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 117/235] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:03.734742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754412c-bdfe-4028-9632-355446dca94b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.735245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754412c-bdfe-4028-9632-355446dca94b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.735318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754412c-bdfe-4028-9632-355446dca94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.758273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4754412c-bdfe-4028-9632-355446dca94b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:03.759489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4754412c-bdfe-4028-9632-355446dca94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:03.856836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 119/236] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:03.892163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a164cb4-811c-4690-8a89-0fb9eb879430 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:03.897043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:03.945374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a206202-1d8b-44a5-bfad-e885f6314219 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.946529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a206202-1d8b-44a5-bfad-e885f6314219 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.946817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a206202-1d8b-44a5-bfad-e885f6314219 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.948229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 120/237] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:03.953799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:03.954035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:03.954322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:03.974743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:03.975949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:04.028313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:04.089732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a164cb4-811c-4690-8a89-0fb9eb879430 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.094162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 121/238] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:04.122214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.173810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.174629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.174918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.175798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.202937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:04.224001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:04.224953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:04.225756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:04.227625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 118/239] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:04.234907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.293682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f937dc-3b02-45d9-b098-19d163672759 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.342975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.343803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.344203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.344898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.346731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:04.348874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.350803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 122/240] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:04.357837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.367073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:04.415739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:04.435008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653178-cd8c-4944-b62d-f78135adb029 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.500145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f825a-a62d-4cce-9700-f2fcb2a00b17 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.504519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.504751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.504870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.505166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.505941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:04.506255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:04.507656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.525103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 119/241] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:04.532260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.552975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ae12-9320-4e0f-91a5-96e876f60133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.553661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ae12-9320-4e0f-91a5-96e876f60133 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.553945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ae12-9320-4e0f-91a5-96e876f60133 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.554217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ae12-9320-4e0f-91a5-96e876f60133 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.554941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894ae12-9320-4e0f-91a5-96e876f60133 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:04.556408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894ae12-9320-4e0f-91a5-96e876f60133 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.595101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 123/242] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:04 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:04.699273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d92d3-5ae6-4c15-a8e6-889d179eba26 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.733272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:04.742605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:04.743522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.743814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.744099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] PATH_INFO: `/v3/users/d74ef9ce9dce4fc6ad3ac81bc29a4498` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.747048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85968cd4-e182-4abe-b02b-0360a89e1972 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.747528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85968cd4-e182-4abe-b02b-0360a89e1972 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.747627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85968cd4-e182-4abe-b02b-0360a89e1972 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.749207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 124/243] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:04.762690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a1cdc-d971-4165-9f8c-e23e1ce4fc65 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.762967 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a1cdc-d971-4165-9f8c-e23e1ce4fc65 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.763168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a1cdc-d971-4165-9f8c-e23e1ce4fc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.785691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e21a1cdc-d971-4165-9f8c-e23e1ce4fc65 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:04.787201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e21a1cdc-d971-4165-9f8c-e23e1ce4fc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:04.790865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=d74ef9ce9dce4fc6ad3ac81bc29a4498)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:04.792360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:04.794204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 120/244] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:04 2022] GET /identity/v3/users/d74ef9ce9dce4fc6ad3ac81bc29a4498 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:04.799835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:04.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 125/245] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:04.903600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd94f4-90f6-48bf-ac51-e7cd6ab8b460 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.903939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd94f4-90f6-48bf-ac51-e7cd6ab8b460 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.904171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd94f4-90f6-48bf-ac51-e7cd6ab8b460 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.913718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 126/246] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:04.921578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:04.921893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:04.922157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:04.945217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:04.945539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1674a1-b0fb-4228-a58f-3eca1bfbe0d6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:04.946953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:04.978784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:05.002102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.002770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.003051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.003996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.044036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 127/247] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:05.045617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.076215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:05.101854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:05.103438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:05.193228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2dab8-f877-4148-a95c-95956ae880e5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:05.239052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.240195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.240625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.241103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.246256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 121/248] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:14:05.247211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.268459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:05.287912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:05.289072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:05.290144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:05.295376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 128/249] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:05.296358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.347422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:05.453016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b4e85-0a49-458b-8ca8-394b29233d25 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:05.457967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:05.498065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.498947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.499237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.499511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.500700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:05.502333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:05.504406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 129/250] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:05.506716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a90c4-f5fc-42c9-93d9-7b3495b7d7d9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:05.508970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.582212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.582916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.583195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.583463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.605369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:05.613802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:05.614772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:05.615567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 122/251] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:05.616268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.674100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377e66d-c8f0-4285-a7ec-2d2b8f3d74d3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:05.677331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:05.722700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.723830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.724312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.724780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.726066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:05.726587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:05.728157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:05.737516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 130/252] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:05.738927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.752460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101c230-5503-4dea-b3c1-28a6e3fd5696 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:05.790385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.791052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.791342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.791624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.792466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] RBAC: Adding query filter params (role.id=41a64573139b4495837c399b914a63b3, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=d74ef9ce9dce4fc6ad3ac81bc29a4498) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:05.792784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:05.794277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:05.800721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 123/253] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/role_assignments?user.id=d74ef9ce9dce4fc6ad3ac81bc29a4498&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=41a64573139b4495837c399b914a63b3 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:05.801650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.814111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:05.855231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d94922-dd4c-4c64-88a6-e279c9b8b622 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:05.918222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:05.919321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.919806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.920295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.921314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:05.921832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:05.924040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d803f-8b94-4386-87b0-67412dcb2570 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:05.938433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 131/254] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:05.940252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851979-10c3-4ba7-8854-cc7a1e14b2f7 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:05.941500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851979-10c3-4ba7-8854-cc7a1e14b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:05.942031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851979-10c3-4ba7-8854-cc7a1e14b2f7 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:05.943642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 132/255] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:05.953997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851979-10c3-4ba7-8854-cc7a1e14b2f7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:05.985294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.053334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e98d-4010-4349-bf3f-72cb15eb5d75 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.103469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b871472-c1c1-45e3-95e4-5dd95f33815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.104805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871472-c1c1-45e3-95e4-5dd95f33815f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.105125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871472-c1c1-45e3-95e4-5dd95f33815f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.105463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871472-c1c1-45e3-95e4-5dd95f33815f None admin] PATH_INFO: `/v3/users/93d97f20e7f147119f0c2b1744055253` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.106911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b871472-c1c1-45e3-95e4-5dd95f33815f None admin] RBAC: Authorizing `identity:get_user(user_id=93d97f20e7f147119f0c2b1744055253)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:06.108762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b871472-c1c1-45e3-95e4-5dd95f33815f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:06.111205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 124/256] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/users/93d97f20e7f147119f0c2b1744055253 => generated 297 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:06.113267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.113556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.113839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.135183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:06.136903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:06.148230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851979-10c3-4ba7-8854-cc7a1e14b2f7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.192782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.194082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.194558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.195023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] PATH_INFO: `/v3/users/935bae62b9f0453fb193f7611f39896d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.196475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] RBAC: Authorizing `identity:get_user(user_id=935bae62b9f0453fb193f7611f39896d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:06.198420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:06.200394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 133/257] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:05 2022] GET /identity/v3/users/935bae62b9f0453fb193f7611f39896d => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:06.201491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.269150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 125/258] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:06.273678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.300897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.345129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.371079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab86fb-541f-489c-914c-935da5f2691e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.441890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.442783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.443069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.443347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.449779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:06.451267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:06.453088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 134/259] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:06.454015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.461276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30998124-7edf-4cb6-9b36-758e70cab8e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.554028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.554691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.554974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.555247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.682975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893c14-df4a-4f3a-8329-cc47589a804f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:06.697436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:06.698939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:06.713245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.747524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:06.748699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:06.749190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:06.749643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:06.791949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:06.803277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:06.804430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:06.805288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 135/260] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:06.811453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.829276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:06.883752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 126/261] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:06.884804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:06.994554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcdbc6-f6a9-4335-8557-121da2937607 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.017269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.034934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.036342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.036949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.037401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.038308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:07.038759 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.040281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.070277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 127/262] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:07.071272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.131106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fffbd0-ceb5-4330-8d21-40d806a59f41 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.207367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.208268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.208554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.208832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/93d97f20e7f147119f0c2b1744055253/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.212415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c6b6badab543ceb85ba21c445514a4, user_id=93d97f20e7f147119f0c2b1744055253, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.213910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.245232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 136/263] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:06 2022] PUT /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/93d97f20e7f147119f0c2b1744055253/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:07.246830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.299129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8547b7-4f67-413c-b877-1b5b7d22c6ae None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.408899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.414723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.415829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.416261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.416708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.421771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.423477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.426344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 128/264] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:07.446204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.469531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509c02-9290-417d-8aa3-027d00246400 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.507553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.508366 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.508654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.508964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] PATH_INFO: `/v3/users/935bae62b9f0453fb193f7611f39896d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.510212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] RBAC: Authorizing `identity:get_user(user_id=935bae62b9f0453fb193f7611f39896d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.511696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.513740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 137/265] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/users/935bae62b9f0453fb193f7611f39896d => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:07.526428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:07.554507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.637281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e75b4-687d-483c-9f1c-fa1934e1e027 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.640482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56fb25-65f2-4706-8ac3-b772307c9b6f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:07.679089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.680242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.681047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.681340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.681722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.683221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=935bae62b9f0453fb193f7611f39896d) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:07.683537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.684655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:07.685017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb9db-87b2-4fc0-b94c-0b43a18428c8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.685519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.686048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.686450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.693132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 129/266] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/role_assignments?user.id=935bae62b9f0453fb193f7611f39896d&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:07.694455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4bc72-b581-4303-8de7-6fb64a1d8072 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.694745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4bc72-b581-4303-8de7-6fb64a1d8072 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.695023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4bc72-b581-4303-8de7-6fb64a1d8072 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.696265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 130/267] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:07.697369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838252b5-3ed8-4b1d-93e2-cfc7b016c80e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.697654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838252b5-3ed8-4b1d-93e2-cfc7b016c80e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.697928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838252b5-3ed8-4b1d-93e2-cfc7b016c80e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.699435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 131/268] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:07.712431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b309d-9cee-4c4c-9542-8fa8aa1c5652 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.712727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b309d-9cee-4c4c-9542-8fa8aa1c5652 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.713032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b309d-9cee-4c4c-9542-8fa8aa1c5652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.735865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e21b309d-9cee-4c4c-9542-8fa8aa1c5652 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:07.737643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e21b309d-9cee-4c4c-9542-8fa8aa1c5652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:07.828990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 132/269] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:07.830975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:07.831713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:07.832422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:07.862099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:07.864590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:07.904038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:07.922883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:07.923947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:07.965602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 133/270] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:07.967274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.043869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 138/271] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:14:08.044787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.098360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.100968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481ff7a-eb51-4352-bf7f-a5eff8b63e46 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.141579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.142684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.143149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.143659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.158336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:08.166892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:08.167815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:08.168625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:08.172085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 134/272] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:08.173232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.186720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50328c2-3a93-4154-b88d-ba99b3eb3578 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.199784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.238616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.239763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.240267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.240729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.242322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.243987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.246251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 139/273] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.247215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.308858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db362-c508-4a18-9b28-d8cf82b9681b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.354729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.356031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.356524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.356995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.358495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.360785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.362858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.364830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 135/274] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.365920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.367728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591ff64-2494-4fc6-b947-3301dfff1a5e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.410171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.410846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.411128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.411398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.412141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:08.412442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.413844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.422404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 140/275] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.423404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.478190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.492316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d13ed-d750-46ec-a450-15cc9dd531b4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.528858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2b67e-d235-4a6e-b3dd-f9bffa0ee712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.529557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2b67e-d235-4a6e-b3dd-f9bffa0ee712 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.529847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2b67e-d235-4a6e-b3dd-f9bffa0ee712 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.530124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2b67e-d235-4a6e-b3dd-f9bffa0ee712 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.530832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2b67e-d235-4a6e-b3dd-f9bffa0ee712 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.532290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2b67e-d235-4a6e-b3dd-f9bffa0ee712 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.538023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab1cb3-2d95-4688-b62e-44e6a9760a06 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.554393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 136/276] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:08 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:08.555761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88bcf6-a989-424b-967e-f44749f7a926 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.556057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88bcf6-a989-424b-967e-f44749f7a926 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.556627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88bcf6-a989-424b-967e-f44749f7a926 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.558104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 137/277] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:08.558711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88bcf6-a989-424b-967e-f44749f7a926 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.594015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.595559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.596038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.597532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.598785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] RBAC: Adding query filter params (role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=935bae62b9f0453fb193f7611f39896d) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:08.599751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.602113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42980f9d-360a-487e-90d1-caf19eb23596 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.611195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 141/278] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/role_assignments?user.id=935bae62b9f0453fb193f7611f39896d&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.614239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.614750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.615731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.644587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:08.646255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:08.677458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.725577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88bcf6-a989-424b-967e-f44749f7a926 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.774653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 142/279] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:08.776356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.777293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.777584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.778125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] PATH_INFO: `/v3/users/6a105c54f6fa4afdbcea103fc478f038` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.794028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] RBAC: Authorizing `identity:get_user(user_id=6a105c54f6fa4afdbcea103fc478f038)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.795559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.797876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 138/280] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/users/6a105c54f6fa4afdbcea103fc478f038 => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.803925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.809928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:08.814397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:08.930953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34cff2-ff50-45b2-a7ae-5c9e6503d25e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.936639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f064aa-8768-4a0c-b1b5-e7318e930c47 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:08.974323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.975150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.975440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.975717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:08.976561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, user.id=6a105c54f6fa4afdbcea103fc478f038, scope.system=all) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:08.976874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:08.978362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:08.983949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 143/281] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/role_assignments?user.id=6a105c54f6fa4afdbcea103fc478f038&scope.system=all&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:08.984939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:08.985918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:08.986208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:08.986488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.000020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:09.008186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:09.009171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:09.009976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 139/282] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:09.017561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:09.042766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:09.139090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adabdda-6157-48e8-9a1f-5fc0701409ed None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:09.178390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:09.179065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:09.179345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:09.179616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.180371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:09.180676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:09.182150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:09.193430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 144/283] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:09.198774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:09.292916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:09.336756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5036830-87cc-4138-aac7-d23115ef416f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:09.373786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:09.374977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:09.375435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:09.375910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] PATH_INFO: `/v3/users/d74ef9ce9dce4fc6ad3ac81bc29a4498` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.377479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] RBAC: Authorizing `identity:get_user(user_id=d74ef9ce9dce4fc6ad3ac81bc29a4498)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:09.379679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:09.381584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 140/284] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:09 2022] GET /identity/v3/users/d74ef9ce9dce4fc6ad3ac81bc29a4498 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:09.390206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:09.512389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:09.538947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02635924-6bf2-4bc0-98ef-7fb2c52c7bb4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:09.598011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dac2e3-46e1-4d9d-ad97-67dd8e97b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:09.599296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dac2e3-46e1-4d9d-ad97-67dd8e97b8be None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:09.599811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dac2e3-46e1-4d9d-ad97-67dd8e97b8be None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:09.600270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dac2e3-46e1-4d9d-ad97-67dd8e97b8be None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.601736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dac2e3-46e1-4d9d-ad97-67dd8e97b8be None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:09.603392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dac2e3-46e1-4d9d-ad97-67dd8e97b8be None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:09.605353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 145/285] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:09 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:09.612256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:09.755669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172bd93-601f-4f73-910d-fdb08a48cf46 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:09.798969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:09.811102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06be06-a456-425a-a511-56f447bf9882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:09.812012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06be06-a456-425a-a511-56f447bf9882 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:09.812300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06be06-a456-425a-a511-56f447bf9882 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:09.812577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06be06-a456-425a-a511-56f447bf9882 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/d74ef9ce9dce4fc6ad3ac81bc29a4498/roles/41a64573139b4495837c399b914a63b3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.814620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06be06-a456-425a-a511-56f447bf9882 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c6b6badab543ceb85ba21c445514a4, user_id=d74ef9ce9dce4fc6ad3ac81bc29a4498, role_id=41a64573139b4495837c399b914a63b3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:09.816822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06be06-a456-425a-a511-56f447bf9882 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:09.838253 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 141/286] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:09 2022] PUT /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4/users/d74ef9ce9dce4fc6ad3ac81bc29a4498/roles/41a64573139b4495837c399b914a63b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:09.935872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6e8b-5c9a-4173-88ba-6938373841f6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:09.936212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6e8b-5c9a-4173-88ba-6938373841f6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:09.936481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6e8b-5c9a-4173-88ba-6938373841f6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.938360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 146/287] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:09.948902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:09.949423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:09.949534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:09.975124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:09.976390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:10.078186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 142/288] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:10.106455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6e8b-5c9a-4173-88ba-6938373841f6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:10.265767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6e8b-5c9a-4173-88ba-6938373841f6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:10.312654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:10.313855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.314343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.314852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.330509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:10.339882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:10.340794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:10.346090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:10.349900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 147/289] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:10.357658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:10.542810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503ef13-a60f-43f2-9cb1-5d4821b5ff13 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:10.570857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:10.591780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:10.592398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.592687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.592994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.593790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:10.594100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:10.595551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:10.609318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 143/290] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:10.616372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:10.636150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:10.788710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d1184-520a-4332-aea7-1c9ea8a0eca4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:10.841672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:10.842661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:10.842928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:10.843152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] PATH_INFO: `/v3/users/93d97f20e7f147119f0c2b1744055253` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:10.845048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] RBAC: Authorizing `identity:get_user(user_id=93d97f20e7f147119f0c2b1744055253)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:10.847156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:10.849441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 148/291] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/users/93d97f20e7f147119f0c2b1744055253 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:10.857449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:11.007206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e53b71-cf97-4566-9a07-a2ba49e0e728 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:11.051111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:11.051776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.052064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.052343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.123509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:11.152343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:11.154125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:11.302352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 144/292] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:11.308014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:11.444591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:11.475889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853aa1d-47b8-4ddc-aa5d-2e52bcf5ee92 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:11.528447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c12afb-d774-49ef-909f-ade83f4bcd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:11.529369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12afb-d774-49ef-909f-ade83f4bcd60 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.529656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12afb-d774-49ef-909f-ade83f4bcd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.529935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12afb-d774-49ef-909f-ade83f4bcd60 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.531176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c12afb-d774-49ef-909f-ade83f4bcd60 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:11.532652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c12afb-d774-49ef-909f-ade83f4bcd60 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:11.534490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 149/293] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:11 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:11.542123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:11.642980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:11.672744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e5a69-2957-4e87-ac8b-0ec3f93a4379 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:11.719477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:11.720150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.720430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.720707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.721601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=93d97f20e7f147119f0c2b1744055253) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:11.722060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:11.723498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67d65-e9cf-408a-bb5b-26983b2f3aeb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:11.733876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 145/294] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:11 2022] GET /identity/v3/role_assignments?user.id=93d97f20e7f147119f0c2b1744055253&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:11.824953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c852d64-3cab-4430-bffd-f690ef28747e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.825299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c852d64-3cab-4430-bffd-f690ef28747e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.825532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c852d64-3cab-4430-bffd-f690ef28747e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.826983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 150/295] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:11.833492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.833786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.834055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.861755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:11.863028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:11.914447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8fa8-df49-4645-8d30-a183c58ed1e8 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.914754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8fa8-df49-4645-8d30-a183c58ed1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.915016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8fa8-df49-4645-8d30-a183c58ed1e8 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.916422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 151/296] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:11.922641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6176ef-3673-4753-8734-0bbf64a8648e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:11.922837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6176ef-3673-4753-8734-0bbf64a8648e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:11.923114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6176ef-3673-4753-8734-0bbf64a8648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:11.958583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fb6176ef-3673-4753-8734-0bbf64a8648e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:11.959713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fb6176ef-3673-4753-8734-0bbf64a8648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:11.973033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 146/297] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:11.996041 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.048703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 152/298] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:12.051148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b4935-34b8-4712-b67d-59379f89298c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.051669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b4935-34b8-4712-b67d-59379f89298c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.052057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b4935-34b8-4712-b67d-59379f89298c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.053668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 153/299] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:12.059225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.059532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.059787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.078106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:12.079327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:12.103999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.141398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d104c-0fc7-4af5-8a9e-69e918d60411 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.165996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 154/300] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:12.173687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.186510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.189152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22435057-1793-441f-b66b-2566995819f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.189974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22435057-1793-441f-b66b-2566995819f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.190284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22435057-1793-441f-b66b-2566995819f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.191099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22435057-1793-441f-b66b-2566995819f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.203076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22435057-1793-441f-b66b-2566995819f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-22435057-1793-441f-b66b-2566995819f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:12.211596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:12.212503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:12.213336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 147/301] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:12.213972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22435057-1793-441f-b66b-2566995819f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.300200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fec9-bb41-4869-97a3-6f2efe344480 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.324725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22435057-1793-441f-b66b-2566995819f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.341506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.342321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.342633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.342912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.354094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:12.361930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:12.362792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:12.363581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 155/302] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:12.364228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.370282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.371336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.371794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.372066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.373347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.374793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:12.376531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 148/303] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:12.381045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.387046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.434845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.471171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2cc9f-6496-47d0-9557-8102e5650119 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.491057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe32127-8798-430c-9648-6a93315e9e88 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.509372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.510051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.510339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.510617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.511379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:12.511688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.513161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:12.522333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 156/304] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:12.523119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.530039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.530706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.530987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.531259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.532365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:12.532365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.533692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfc774-bfb1-4c71-a9c3-b8891dfab404 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:12.542090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 149/305] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:12.543357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348b94-d612-4492-b935-333f2baca090 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.543648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348b94-d612-4492-b935-333f2baca090 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.543920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348b94-d612-4492-b935-333f2baca090 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.545459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 150/306] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:12.545909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348b94-d612-4492-b935-333f2baca090 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.615286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.631819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ab9aa-d0e3-4c0c-86e9-0fa7f9dec9e4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.652538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348b94-d612-4492-b935-333f2baca090 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.660072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.667921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.668584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.668864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.669162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] PATH_INFO: `/v3/groups` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.669984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.671592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:12.689173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 157/307] 10.222.0.79 () {60 vars in 1134 bytes} [Wed Jul 20 19:14:12 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:12.689173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.689782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b13038-70ef-4d3e-b1cb-37734ef5d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.691216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13038-70ef-4d3e-b1cb-37734ef5d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.691607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13038-70ef-4d3e-b1cb-37734ef5d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.691917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13038-70ef-4d3e-b1cb-37734ef5d8b0 None admin] PATH_INFO: `/v3/users/778eb57182af4dfa8666319bef802ddd` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.708371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b13038-70ef-4d3e-b1cb-37734ef5d8b0 None admin] RBAC: Authorizing `identity:get_user(user_id=778eb57182af4dfa8666319bef802ddd)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.709920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b13038-70ef-4d3e-b1cb-37734ef5d8b0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:12.711963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 151/308] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/users/778eb57182af4dfa8666319bef802ddd => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:12.718058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.718350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.718632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.737858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:12.739393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:12.824147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 152/309] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:12.826129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.828765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02597156-7466-424c-a7bd-1db2f8460d9e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.876191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.877514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.877856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.878679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] PATH_INFO: `/v3/users/6a105c54f6fa4afdbcea103fc478f038` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.879911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] RBAC: Authorizing `identity:get_user(user_id=6a105c54f6fa4afdbcea103fc478f038)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.881451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:12.886764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 158/310] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/users/6a105c54f6fa4afdbcea103fc478f038 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:12.886764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:12.900939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.952831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e12e3-7d8d-4f03-9d05-dab12bc62355 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:12.953941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:12.993666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:12.994249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:12.994538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:12.994816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:12.995670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] RBAC: Adding query filter params (role.id=41a64573139b4495837c399b914a63b3, user.id=778eb57182af4dfa8666319bef802ddd, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:12.995990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:12.997713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:13.003284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 153/311] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/role_assignments?user.id=778eb57182af4dfa8666319bef802ddd&scope.system=all&role.id=41a64573139b4495837c399b914a63b3 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:13.003984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:13.004373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24658834-767a-4fc0-ad22-31180e50fb48 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:13.052789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:13.053764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.054106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.054448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.067780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:13.076688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:13.077913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:13.078902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:13.080302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 159/312] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:13.087171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:13.154795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6b9fe-3dd9-4099-a2c1-992b9e5bab8c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:13.182552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:13.209904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:13.210805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.211043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.211312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] PATH_INFO: `/v3/system/users/6a105c54f6fa4afdbcea103fc478f038/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.212816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a105c54f6fa4afdbcea103fc478f038, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:13.214382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:13.225869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 154/313] 10.222.0.79 () {58 vars in 1334 bytes} [Wed Jul 20 19:14:13 2022] PUT /identity/v3/system/users/6a105c54f6fa4afdbcea103fc478f038/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:13.236962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dfcf1-b855-48bc-93ca-4ea3470413bd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:13.259963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:13.286211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:13.286933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.287216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.287491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.288293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:13.288604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:13.290147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:13.302872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 160/314] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:13.308681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:13.432528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f526f-c0ca-4bb9-adf0-e63e0a6ddc80 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:13.473519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:13.474406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.474691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.474965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] PATH_INFO: `/v3/users/d74ef9ce9dce4fc6ad3ac81bc29a4498` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.476188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] RBAC: Authorizing `identity:get_user(user_id=d74ef9ce9dce4fc6ad3ac81bc29a4498)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:13.477684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:13.479495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 155/315] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:13 2022] GET /identity/v3/users/d74ef9ce9dce4fc6ad3ac81bc29a4498 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:13.484990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:13.598165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa95c0-1d9f-474e-bf67-b9220bef5bfd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:13.637365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:13.638256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:13.638570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:13.638847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:13.700313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:13.732757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:13.734489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:13.852404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 161/316] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:13.858123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:13.919834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:13.977330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68952ead-3948-400a-afe1-ea39b7cc2004 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:14.015893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1cc78-31e2-438f-be1f-9c5509a51f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:14.017186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1cc78-31e2-438f-be1f-9c5509a51f07 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:14.017668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1cc78-31e2-438f-be1f-9c5509a51f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:14.018135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1cc78-31e2-438f-be1f-9c5509a51f07 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:14.019547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1cc78-31e2-438f-be1f-9c5509a51f07 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:14.021228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1cc78-31e2-438f-be1f-9c5509a51f07 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:14.023120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 156/317] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:13 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:14.029984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:14.119511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:14.143429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5f3d-e8af-49a3-9eb4-1f7afe586ff2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:14.179379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:14.180039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:14.180321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:14.180591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:14.181457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] RBAC: Adding query filter params (role.id=41a64573139b4495837c399b914a63b3, scope.project.id=c5c6b6badab543ceb85ba21c445514a4, user.id=d74ef9ce9dce4fc6ad3ac81bc29a4498) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:14.181766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:14.183335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c1cb0-63db-4a3d-9b5c-4f43a3d427ae None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:14.191084 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 162/318] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:14 2022] GET /identity/v3/role_assignments?user.id=d74ef9ce9dce4fc6ad3ac81bc29a4498&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=41a64573139b4495837c399b914a63b3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:14.942351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038424b-cb7a-475d-93da-e327c1f52850 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:14.942863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038424b-cb7a-475d-93da-e327c1f52850 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:14.942970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038424b-cb7a-475d-93da-e327c1f52850 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:14.944325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 157/319] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:14.949718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7268dd6-777d-4eae-aa8e-e7c72b34d5cc None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:14.950103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7268dd6-777d-4eae-aa8e-e7c72b34d5cc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:14.950259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7268dd6-777d-4eae-aa8e-e7c72b34d5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:14.967911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e7268dd6-777d-4eae-aa8e-e7c72b34d5cc None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:14.969142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e7268dd6-777d-4eae-aa8e-e7c72b34d5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:15.040529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 163/320] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:15.064197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038424b-cb7a-475d-93da-e327c1f52850 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.133478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e9dcc-d732-4e59-bd0a-0e29fb04db1a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.133926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e9dcc-d732-4e59-bd0a-0e29fb04db1a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.134260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e9dcc-d732-4e59-bd0a-0e29fb04db1a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.136463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 164/321] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:15.142080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.142372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.142656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.161066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:15.162266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:15.173870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038424b-cb7a-475d-93da-e327c1f52850 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.193511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.215506 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.216741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.217080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.217362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.218539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.220443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.222308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 158/322] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:15.227796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.235633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 165/323] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:15.259371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.261537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.340992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbd50-e81a-49fa-835e-eb58e07a8a2d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.375416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.376086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.376190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bc590-1d8b-4ca1-a1e1-d42a839360fc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.376371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.376647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] PATH_INFO: `/v3/groups` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.377373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.379326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.404910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 159/324] 10.222.0.79 () {60 vars in 1134 bytes} [Wed Jul 20 19:14:15 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:14:15.426771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384a39-9c71-4af1-8f8d-ae7f195ab3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.427608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384a39-9c71-4af1-8f8d-ae7f195ab3cd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.427893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384a39-9c71-4af1-8f8d-ae7f195ab3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.428170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384a39-9c71-4af1-8f8d-ae7f195ab3cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.440957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a384a39-9c71-4af1-8f8d-ae7f195ab3cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-6a384a39-9c71-4af1-8f8d-ae7f195ab3cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:15.449163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:15.450235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:15.451063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 166/325] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:15.457324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.467670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a21b6a-5e91-42ec-b62a-aed4abea1270 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.467954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a21b6a-5e91-42ec-b62a-aed4abea1270 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.468160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a21b6a-5e91-42ec-b62a-aed4abea1270 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.469751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 167/326] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:15.475597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.475887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.476163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.494913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:15.496207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:15.511894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.568311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.580547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 168/327] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:15.580710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8ddb8-7df2-4a27-ae58-b6bea66a64c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.604985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.623717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.623717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.623717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.624025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.625184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:15.625184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.626251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.636686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 160/328] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:15.642513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.739000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecd08e-1e6f-4f66-b8ef-92bbfd54ff00 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.779585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257d781-578c-4f71-aada-ffc89ee8d2f5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.783633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.785189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.785490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.785739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.793345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.797231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:15.807805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:15.808748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:15.809559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 169/329] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:15.815842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.831554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.832816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.833137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.833418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] PATH_INFO: `/v3/users/778eb57182af4dfa8666319bef802ddd` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.834677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] RBAC: Authorizing `identity:get_user(user_id=778eb57182af4dfa8666319bef802ddd)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.836125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.840923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 161/330] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/users/778eb57182af4dfa8666319bef802ddd => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:15.844661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:15.884111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:15.943837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe589219-45d4-47af-aea0-94eaaed208aa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.975396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3cb6-1621-4197-a94d-f4ffbc7620ba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:15.982365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:15.983555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:15.984016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:15.984501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:15.985548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:15.986008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:15.988444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:15.997946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 170/331] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:16.004425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:16.017966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:16.019562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:16.020072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:16.020370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] PATH_INFO: `/v3/system/users/778eb57182af4dfa8666319bef802ddd/roles/41a64573139b4495837c399b914a63b3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:16.024480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=778eb57182af4dfa8666319bef802ddd, role_id=41a64573139b4495837c399b914a63b3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:16.026206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:16.037471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 162/332] 10.222.0.79 () {58 vars in 1334 bytes} [Wed Jul 20 19:14:15 2022] PUT /identity/v3/system/users/778eb57182af4dfa8666319bef802ddd/roles/41a64573139b4495837c399b914a63b3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:16.134241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d2a7c-f74c-4816-ab0e-074ce89923fb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:16.150846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:16.175568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1639b51-4903-4393-a281-1204a9455a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:16.176784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1639b51-4903-4393-a281-1204a9455a97 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:16.177287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1639b51-4903-4393-a281-1204a9455a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:16.177745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1639b51-4903-4393-a281-1204a9455a97 None admin] PATH_INFO: `/v3/users/6a105c54f6fa4afdbcea103fc478f038` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:16.179360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1639b51-4903-4393-a281-1204a9455a97 None admin] RBAC: Authorizing `identity:get_user(user_id=6a105c54f6fa4afdbcea103fc478f038)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:16.181085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1639b51-4903-4393-a281-1204a9455a97 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:16.182986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 171/333] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:16 2022] GET /identity/v3/users/6a105c54f6fa4afdbcea103fc478f038 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:16.189093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:16.241130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:16.313917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4a99-53ae-49f6-af12-3521c81014e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:16.356850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:16.357536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:16.357828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:16.358101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:16.358945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] RBAC: Adding query filter params (role.id=cceb4e5cab5449d893b958c6130869e3, user.id=6a105c54f6fa4afdbcea103fc478f038, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:16.359261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:16.360721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48476c-1b30-4a51-8384-226d563c6620 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:16.374236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 163/334] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Jul 20 19:14:16 2022] GET /identity/v3/role_assignments?user.id=6a105c54f6fa4afdbcea103fc478f038&scope.system=all&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:17.805322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341de1-b2ff-4f56-8479-a3f7353e6361 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:17.806309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341de1-b2ff-4f56-8479-a3f7353e6361 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:17.806790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341de1-b2ff-4f56-8479-a3f7353e6361 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:17.808479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 172/335] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:17.819028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:17.819910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:17.820381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:17.845891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:17.848222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:17.926276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 164/336] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:17.964496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341de1-b2ff-4f56-8479-a3f7353e6361 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.086513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341de1-b2ff-4f56-8479-a3f7353e6361 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.123973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba417-ca36-4a33-a366-5e56231ccdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.124781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba417-ca36-4a33-a366-5e56231ccdbb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.125122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba417-ca36-4a33-a366-5e56231ccdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.125399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba417-ca36-4a33-a366-5e56231ccdbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.141109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba417-ca36-4a33-a366-5e56231ccdbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7f8ba417-ca36-4a33-a366-5e56231ccdbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:18.149660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:18.150567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:18.151378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 173/337] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:18.158582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.201187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c840b-aaf1-4222-b08d-787e02e0d73b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.201524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c840b-aaf1-4222-b08d-787e02e0d73b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.201748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c840b-aaf1-4222-b08d-787e02e0d73b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.203368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 174/338] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:18.208758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.209041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.209290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.209571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.228284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:18.229426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:18.280301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.312077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae54e3-23a6-4cbe-adce-e22c55cc4495 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.337311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 175/339] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:18.367443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.368453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.369015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.369489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.370433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:18.370937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:18.372512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:18.388427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 165/340] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:18.393748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.394658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.540440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d96b9-94aa-4059-b8cf-4de20877075e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.585418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.586086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.586528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.586815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.589271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.639723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba166ae7-5bbf-4728-970e-30afa47d6a0e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.649432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.684000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:18.685520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:18.723630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.724823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.725350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.725825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.754473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:18.763094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:18.763962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:18.764762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:18.773169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 166/341] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:18.780419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.797951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 176/342] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:18.804438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:18.882197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:18.916835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97370f4-05c4-4e90-8158-a3b717e99085 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:18.960392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:18.961295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:18.961578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:18.961851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:18.965637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:18.967296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:18.969442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 177/343] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:18.975682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:19.028929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358bb1-17d3-4370-9830-253b82d61d3b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:19.101026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:19.102054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.104450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.104968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.105943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:19.107921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:19.109580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2880-e056-44bc-91e0-736fde15dccb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:19.132106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 167/344] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:19.137835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6019-eb00-4d52-9129-d8ce21f28bd7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.139555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6019-eb00-4d52-9129-d8ce21f28bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.140815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6019-eb00-4d52-9129-d8ce21f28bd7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.145846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 168/345] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:19.151741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a8028-69d4-4922-a42c-e42eaa0f7456 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:19.158102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6019-eb00-4d52-9129-d8ce21f28bd7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:19.183415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:19.257592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02ab99-9a54-4609-b649-8cceb1c8e8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:19.258464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02ab99-9a54-4609-b649-8cceb1c8e8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.258742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02ab99-9a54-4609-b649-8cceb1c8e8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.259011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02ab99-9a54-4609-b649-8cceb1c8e8e5 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.278152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02ab99-9a54-4609-b649-8cceb1c8e8e5 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:19.279648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02ab99-9a54-4609-b649-8cceb1c8e8e5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:19.281446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 178/346] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:18 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:19.282655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.282952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.283213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.302810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:19.307102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6019-eb00-4d52-9129-d8ce21f28bd7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:19.318624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:19.319825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:19.380304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87756424-98b7-4f5d-bdd8-90922e078edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:19.380905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87756424-98b7-4f5d-bdd8-90922e078edb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.381199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87756424-98b7-4f5d-bdd8-90922e078edb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.381474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87756424-98b7-4f5d-bdd8-90922e078edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.446351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 179/347] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:19.448056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:19.499010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87756424-98b7-4f5d-bdd8-90922e078edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:19.500816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87756424-98b7-4f5d-bdd8-90922e078edb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:19.533294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:19.587756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caca0d0-91a2-4382-8f7e-537717fd78f5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:19.630956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:19.631154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 169/348] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:19.631565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.631857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.632184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.632587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7010-4407-447e-b90b-951db1b4123c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.632904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7010-4407-447e-b90b-951db1b4123c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.633097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=3f00ea992c824af582c7fbffe696728b) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:19.633206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7010-4407-447e-b90b-951db1b4123c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.633420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:19.634873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a1d9b-438d-49f6-b019-74ea1d3c4650 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:19.639729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:19.641017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 170/349] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:19.641959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef7010-4407-447e-b90b-951db1b4123c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:19.647874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 180/350] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:19 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:19.649244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.649536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.649811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.674001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:19.675821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:19.816054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 181/351] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:19.818349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:19.841548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef7010-4407-447e-b90b-951db1b4123c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:19.859029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:19.896198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:19.897657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:19.898152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:19.898579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:19.915397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:19.927025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:19.928170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:14:19.929157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:19.931893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 171/352] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:19.933062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.025394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860312-e885-47fb-9897-6a565acd5424 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.085286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:20.098862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.099742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.100029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.100302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.103804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:20.105405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:20.107167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 182/353] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:19 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:20.107937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.135671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda472f-301f-42c8-a067-f640ec915449 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.210676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.211846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.212332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.212820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.229304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:20.242132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:20.243062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:20.243840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:20.246416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 172/354] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:20.247538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.283408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a001e8-7ac3-4ae8-ae1c-6daf2458271e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.291973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:20.326262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af3198-aa49-4caa-950b-93003461755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.327311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3198-aa49-4caa-950b-93003461755c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.327818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3198-aa49-4caa-950b-93003461755c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.328337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3198-aa49-4caa-950b-93003461755c None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.329419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af3198-aa49-4caa-950b-93003461755c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:20.329923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af3198-aa49-4caa-950b-93003461755c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:20.332645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af3198-aa49-4caa-950b-93003461755c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:20.343691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 183/355] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:20.345544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062459-b259-4e50-ab2b-e1a89e53b493 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.346045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062459-b259-4e50-ab2b-e1a89e53b493 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.347397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062459-b259-4e50-ab2b-e1a89e53b493 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.349935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 184/356] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:20.351434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb062459-b259-4e50-ab2b-e1a89e53b493 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.372208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e012260-1c36-4a43-883f-9d522c73cbc5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.411465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aacac5-af4f-4e20-971a-f40ca8bf5ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.413597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aacac5-af4f-4e20-971a-f40ca8bf5ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.413597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aacac5-af4f-4e20-971a-f40ca8bf5ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.413597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aacac5-af4f-4e20-971a-f40ca8bf5ac3 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.414137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aacac5-af4f-4e20-971a-f40ca8bf5ac3 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:20.415572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aacac5-af4f-4e20-971a-f40ca8bf5ac3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:20.418331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 173/357] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:20.420721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0f31-a596-4deb-8215-f4eec0d32ce0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.420721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0f31-a596-4deb-8215-f4eec0d32ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.420721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0f31-a596-4deb-8215-f4eec0d32ce0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.422156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 174/358] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:20.423026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea0f31-a596-4deb-8215-f4eec0d32ce0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.429877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:20.486110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb062459-b259-4e50-ab2b-e1a89e53b493 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.527456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.528449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.528701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.529017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.529800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:20.530076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:20.531463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a8e3c-3959-4554-bdf7-fca11e6e59e3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:20.545598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 185/359] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:20.546340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.546631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.546904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.565317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:20.566556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:20.576225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:20.579827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea0f31-a596-4deb-8215-f4eec0d32ce0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.625743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb4ad6-7689-41ea-b402-bafd3f44af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.626882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb4ad6-7689-41ea-b402-bafd3f44af34 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.627167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb4ad6-7689-41ea-b402-bafd3f44af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.627482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb4ad6-7689-41ea-b402-bafd3f44af34 None admin] PATH_INFO: `/v3/users/778eb57182af4dfa8666319bef802ddd` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.631257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb4ad6-7689-41ea-b402-bafd3f44af34 None admin] RBAC: Authorizing `identity:get_user(user_id=778eb57182af4dfa8666319bef802ddd)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:20.631865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb4ad6-7689-41ea-b402-bafd3f44af34 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:20.634069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 175/360] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/users/778eb57182af4dfa8666319bef802ddd => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:20.634800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.635088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.635368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.656995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:20.658427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:20.664587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 186/361] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:20.665887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.727697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:20.773955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 176/362] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:20.774728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.797327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c17c9-ce7f-434f-8075-45bc6eb6cd9a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.839300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.839971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.840254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.840535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.841463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=3f00ea992c824af582c7fbffe696728b) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:20.841779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:20.843231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:20.852913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 187/363] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:20.852913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:20.918950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:20.925454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72064f7-ae65-4bad-b85d-f860e3d76a82 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:20.974399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:20.975732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:20.976271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:20.976936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:20.985910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12183202-40e1-4f81-862f-3bb73f3f0dad None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.005722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:21.038895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.039540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.039836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.040111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.041030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] RBAC: Adding query filter params (role.id=41a64573139b4495837c399b914a63b3, user.id=778eb57182af4dfa8666319bef802ddd, scope.system=all) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:21.041348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:21.042817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:21.058485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 188/364] 10.222.0.79 () {56 vars in 1308 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/role_assignments?user.id=778eb57182af4dfa8666319bef802ddd&scope.system=all&role.id=41a64573139b4495837c399b914a63b3 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:21.059415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.077865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:21.080270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:21.205137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:21.216055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 177/365] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:21.217580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.324778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6d39f-b872-4486-ba60-ea0571db35c7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.358762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b73cb-648e-4d31-8793-d658c1885e33 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.378643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.379470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.379761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.380041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.394865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:21.404563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.405705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:21.405897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:21.406795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:21.407651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 189/366] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:21.408364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.410380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.410889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.423504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:21.433349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:21.434364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:21.435197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:21.438081 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 178/367] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:21.439199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.450809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:21.507685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:21.535286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcc797-b7c2-410a-97ef-1f80c3a23477 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.557979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917baa1-aa0f-43e4-b127-fc254871134f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.577297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.578100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.578386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.578666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.579882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:21.582287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:21.584453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 190/368] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:21.585771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.598633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.599656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.600204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.600717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.601777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:21.602349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:21.604017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:21.613771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 179/369] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:21.615482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.729040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3d39e-4bb0-445b-984d-def92d7a45c4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.775228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:21.778252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e004b0-0c32-4ec5-b419-9f95d46b26f7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.780319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.781195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.781396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.781678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.782452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:21.783629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:21.785018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:21.795376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 191/370] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:21.796225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.805882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:21.823547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.824250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.824604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.824990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.826455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:21.828662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:21.831949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 180/371] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:21.832940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:21.925668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd240f0-590b-4b39-9a34-14ff0e848cd4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:21.971791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:21.973094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:21.973603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:21.974101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:21.974945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030a08f-61e1-47be-ae7c-30caf8a8b163 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.013266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.016986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dae16-c593-47bc-b499-ff48c384b30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.017776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dae16-c593-47bc-b499-ff48c384b30a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.018200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dae16-c593-47bc-b499-ff48c384b30a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.018591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dae16-c593-47bc-b499-ff48c384b30a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.053138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.055504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.091423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.108415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dae16-c593-47bc-b499-ff48c384b30a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.109972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dae16-c593-47bc-b499-ff48c384b30a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.199956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 192/372] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:22.200898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.257742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 181/373] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:22.258958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f24a9-1cda-45ad-a4ac-71e17d7660da None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.259239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f24a9-1cda-45ad-a4ac-71e17d7660da None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.259525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f24a9-1cda-45ad-a4ac-71e17d7660da None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.261333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 182/374] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:22.261724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f24a9-1cda-45ad-a4ac-71e17d7660da None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.297582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.348475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3061c-4df2-4523-bde4-5374fb7f38e3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.392675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.394651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.395322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.395609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.395886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.396729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=c5c6b6badab543ceb85ba21c445514a4) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:22.397073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.398509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.409973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 193/375] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:22.409973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.415233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f24a9-1cda-45ad-a4ac-71e17d7660da None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.456789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5b219-a265-4fba-bc4a-1b15c013bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.458064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5b219-a265-4fba-bc4a-1b15c013bb19 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.458586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5b219-a265-4fba-bc4a-1b15c013bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.459046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5b219-a265-4fba-bc4a-1b15c013bb19 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.460666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5b219-a265-4fba-bc4a-1b15c013bb19 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.462467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5b219-a265-4fba-bc4a-1b15c013bb19 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.470325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 183/376] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:22.472330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.472872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.473355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.493692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:22.495213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:22.534864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987396d9-8e43-4656-829f-0da19b9c055c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.564993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.574185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 184/377] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:22.575200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.576189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.577273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.577727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.578154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] PATH_INFO: `/v3/projects/8afe543881644c988270ab8de48728b1` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.580782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=8afe543881644c988270ab8de48728b1)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.583517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.585842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 194/378] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/projects/8afe543881644c988270ab8de48728b1 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:22.591948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.681191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.703542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48d79a-9b27-460c-a277-594c2b629af8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.722303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d47f38-503d-4001-b185-6853b6b13ef5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.744811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.745847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.746237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.746617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.748451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.750866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.752189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 185/379] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:22.753588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.776640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.778959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.779484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.779922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] PATH_INFO: `/v3/groups/bcc0f4a0a8e0453bbdc44ba9d820f233` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.790627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] RBAC: Authorizing `identity:get_group(group_id=bcc0f4a0a8e0453bbdc44ba9d820f233)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.790627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.793541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 195/380] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/groups/bcc0f4a0a8e0453bbdc44ba9d820f233 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:22.793541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.876142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:22.915868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548efa0b-3261-4ee9-9775-8e0074abb9c0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.930100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9617ec-3e1a-47b4-bcd2-12a235bc3065 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:22.958605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.959748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.960231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.960727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.976702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:22.976907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:22.977626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:22.977920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:22.978201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:22.979049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=c5c6b6badab543ceb85ba21c445514a4) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:22.979366 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:22.980821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:22.987053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:22.990267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:22.991083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:22.991700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 196/381] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:22.999033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:22.999594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 186/382] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:23.002135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:23.034827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:23.139386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c784b-e675-4e9a-a3cf-14e268b9080e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:23.177161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:23.195460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:23.196139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.196427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.196706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.197599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] RBAC: Adding query filter params (group.id=bcc0f4a0a8e0453bbdc44ba9d820f233, role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=8afe543881644c988270ab8de48728b1) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:23.197924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:23.199372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:23.206005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 197/383] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:22 2022] GET /identity/v3/role_assignments?group.id=bcc0f4a0a8e0453bbdc44ba9d820f233&scope.project.id=8afe543881644c988270ab8de48728b1&role.id=689534f62373409f9d02ea4acc335ab7 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:23.230992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7301e62-0bec-4e1a-8ee6-153ccbf8c6ea None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:23.271811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:23.272590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.272779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.273095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.273929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:23.274289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:23.275698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54ee0-0e78-4ab1-9a63-6fa79503137b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:23.286816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 187/384] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:23.294708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:23.376548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a46cf-c2f4-4f92-b4ce-a9cc797ff140 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.376906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a46cf-c2f4-4f92-b4ce-a9cc797ff140 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.377143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a46cf-c2f4-4f92-b4ce-a9cc797ff140 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.380000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 188/385] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:23.387774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.388166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.388456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.413691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:23.415083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:23.473174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4214eb-f038-41bc-a6a4-539574c152d7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:23.516556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:23.517387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.517676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.518868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.520479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:23.521987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:23.523702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 198/386] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:23 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:23.530189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:23.533586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 189/387] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:23.604768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:23.613982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:23.624979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:23.681938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c8c61-4143-4a97-9ce6-804f6e6e9a66 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:23.727508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:23.728401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.729322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.729612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.731070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:23.733002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:23.735334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 199/388] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:23 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:23.741669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:23.862059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744efd1-5bee-438c-ab1a-60c669f9147b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:23.879515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ec1e5-b89d-4339-9b82-e640c758c4e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:23.889319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:23.902582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:23.904241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.904536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.904864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.906910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f00ea992c824af582c7fbffe696728b, group_id=fc220d7c057847fdbf2c2cbd6d9216da, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:23.908412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:23.927018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 200/389] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Jul 20 19:14:23 2022] PUT /identity/v3/projects/3f00ea992c824af582c7fbffe696728b/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:14:23.959898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:23.961027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:23.961323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:23.961654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:23.975524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:23.984557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:23.985569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:23.986365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:23.987214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 190/390] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:23.993027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:24.165479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c053ce2-eea5-42c8-a3c2-bb87647b5416 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:24.213971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:24.214657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:24.214939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:24.215214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:24.215984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:24.216290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:24.217716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:24.228171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 201/391] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:24.233909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:24.345252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:24.400706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478096e-a7b3-4849-8363-75eb24fb4eb1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:24.444243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:24.445500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:24.445992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:24.446480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:24.453589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:24.455294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:24.458245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 191/392] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:24 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:24.466138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:24.523099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:24.589136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8fcea-dfa9-4da9-a1f9-3d5d246c95b7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:24.631561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac11b0-cd81-4731-998c-192ba7c6767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:24.632448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac11b0-cd81-4731-998c-192ba7c6767b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:24.632729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac11b0-cd81-4731-998c-192ba7c6767b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:24.633105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac11b0-cd81-4731-998c-192ba7c6767b None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:24.634399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac11b0-cd81-4731-998c-192ba7c6767b None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:24.635930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac11b0-cd81-4731-998c-192ba7c6767b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:24.638033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 202/393] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:24 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:24.644198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:24.778416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568baab1-b67d-4d8b-90db-939bf9f18cc6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:24.820169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a100-d069-494b-b4e3-6268a70119f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:24.821432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a100-d069-494b-b4e3-6268a70119f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:24.821913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a100-d069-494b-b4e3-6268a70119f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:24.822359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a100-d069-494b-b4e3-6268a70119f5 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:24.824830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a100-d069-494b-b4e3-6268a70119f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f00ea992c824af582c7fbffe696728b, group_id=fc220d7c057847fdbf2c2cbd6d9216da, role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:24.826547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a100-d069-494b-b4e3-6268a70119f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:24.829227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:24.846069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 192/394] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Jul 20 19:14:24 2022] PUT /identity/v3/projects/3f00ea992c824af582c7fbffe696728b/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/a0ef0911aac646be8916503dba5a9c3f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:14:25.209652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997be76-a9b8-4a93-8d75-53398e287187 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:25.210198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997be76-a9b8-4a93-8d75-53398e287187 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:25.210198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997be76-a9b8-4a93-8d75-53398e287187 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:25.211844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 203/395] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:25.217480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:25.217965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:25.218396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:25.246678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:25.249178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:25.348373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 193/396] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:25.376644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997be76-a9b8-4a93-8d75-53398e287187 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:25.529744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997be76-a9b8-4a93-8d75-53398e287187 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:25.552697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:25.593000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:25.594699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:25.595359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:25.596126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:25.619283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:25.630734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:25.632249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:25.633305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:25.635857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 204/397] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:25.642800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:25.829367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36314e8-9381-4c8b-a530-69c5bd82dd0f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:25.869120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:25.886569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:25.887974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:25.888444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:25.888740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:25.889549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:25.889866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:25.891375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817edb45-1f7d-4f44-9e48-67ac7ebf5c2b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:25.901598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 194/398] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:25.910398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.058291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446f6d-72a4-4ce0-9dd6-363633c52283 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.058681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446f6d-72a4-4ce0-9dd6-363633c52283 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.058865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446f6d-72a4-4ce0-9dd6-363633c52283 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.060355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 195/399] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:26.066118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8723a9-a36c-496f-b3e2-1772af0a6599 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.066431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8723a9-a36c-496f-b3e2-1772af0a6599 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.066829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8723a9-a36c-496f-b3e2-1772af0a6599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.093478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0e8723a9-a36c-496f-b3e2-1772af0a6599 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:26.094654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0e8723a9-a36c-496f-b3e2-1772af0a6599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:26.097148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c966adc-17d9-41b1-955b-871b66308497 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.148916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f698b2a-b418-454f-ac11-21987a4a8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.150276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f698b2a-b418-454f-ac11-21987a4a8d62 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.150680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f698b2a-b418-454f-ac11-21987a4a8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.150975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f698b2a-b418-454f-ac11-21987a4a8d62 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.153341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f698b2a-b418-454f-ac11-21987a4a8d62 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:26.155060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f698b2a-b418-454f-ac11-21987a4a8d62 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:26.163625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 205/400] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:25 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:26.163625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e06ed-fabd-4e52-a163-5ca532daf00f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.163625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e06ed-fabd-4e52-a163-5ca532daf00f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.163625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e06ed-fabd-4e52-a163-5ca532daf00f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.163625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 206/401] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:26.169290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e06ed-fabd-4e52-a163-5ca532daf00f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.218151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 196/402] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:26.219897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.220400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.220867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.233875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:26.263192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:26.265041 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:26.327251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e06ed-fabd-4e52-a163-5ca532daf00f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.369191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.370001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.370310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.370555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.371786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:26.373551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:26.380908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 207/403] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:26.380908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.384900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 197/404] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:26.386424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.455040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:26.499005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8322e-5b73-46a3-9132-9172b837fecc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.512322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:26.525326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab162ef-eeca-4416-807c-651bc2018d29 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.559479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.560728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.561155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.561551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.565761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.567109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.567571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.567994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.570209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c6b6badab543ceb85ba21c445514a4, group_id=fc220d7c057847fdbf2c2cbd6d9216da, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:26.571988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:26.574655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:26.585174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:26.586195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:26.587026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 208/405] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:26.587793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.595100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 198/406] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Jul 20 19:14:26 2022] PUT /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:26.596712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.724935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f358fa-8518-49f8-81f8-398b1dbff5b9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.725543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605a5bd-5d62-43f2-bb2e-7a95649e002d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.733901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:26.772851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.773566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.773668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.773856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.774185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.775892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.775892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.776059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:26.776138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.776213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:26.778486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb74cd-b552-461b-b3fe-c3ee47036729 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:26.788182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 199/407] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:26.788368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:26.794125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1855b08-cbf6-458b-9bc9-7d2ebdba8395 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.794627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1855b08-cbf6-458b-9bc9-7d2ebdba8395 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.795099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1855b08-cbf6-458b-9bc9-7d2ebdba8395 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:26.796359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:26.797281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:26.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 209/408] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:26.798811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.800242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 200/409] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:26.806346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.806640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.806928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.828870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:26.830140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:26.841823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:26.908426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 201/410] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:26.909746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:26.950287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c55af-92c5-4697-9a2a-40b36b4d2657 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:26.989770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:26.991079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:26.991551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:26.992026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:26.993530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:26.995316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:26.997303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 210/411] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:26.998353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.028663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:27.032412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34565-9b98-45d7-8c80-b84c353f2bee None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.070646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.071503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.071889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.072260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.073131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:27.073505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.074958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.084323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 202/412] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:27.085350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.113539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:27.122030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89417fa-aeff-43f9-9272-c92e04c7164f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.160163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.161363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.161853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.162361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.176272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:27.185523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:27.186441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:27.187262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:27.190636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c3ec4-e719-4a27-9517-13aeab5f3103 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.191419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 211/413] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:27.192422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.235535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.236402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.236806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.237256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.238925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.240397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.242523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 203/414] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:27 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:27.243460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.276758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:27.322670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f038b69-975c-4e3d-ac27-dced0b1b3c62 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.360323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a94b-3c0a-4726-bd4f-40de80c98f78 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.360525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.361344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.361662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.361948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] PATH_INFO: `/v3/groups/bcc0f4a0a8e0453bbdc44ba9d820f233` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.363394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] RBAC: Authorizing `identity:get_group(group_id=bcc0f4a0a8e0453bbdc44ba9d820f233)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.364856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.367230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 212/415] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:27 2022] GET /identity/v3/groups/bcc0f4a0a8e0453bbdc44ba9d820f233 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:27.368197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.398866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.399539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.399824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.400101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.400859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:27.401275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.401429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:27.402993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.412399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 204/416] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:27.417705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.486276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188c84c-39a8-445c-beba-cbc71fa403df None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.522982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:27.527503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.528753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.529078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.529363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.531958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5c6b6badab543ceb85ba21c445514a4, group_id=fc220d7c057847fdbf2c2cbd6d9216da, role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.533700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.534516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd40d31-192c-4c68-a458-2e66193d8953 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.557159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 213/417] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Jul 20 19:14:27 2022] PUT /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4/groups/fc220d7c057847fdbf2c2cbd6d9216da/roles/a0ef0911aac646be8916503dba5a9c3f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:27.558382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.574251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.575162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.575418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.575818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] PATH_INFO: `/v3/projects/8afe543881644c988270ab8de48728b1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.577149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] RBAC: Authorizing `identity:get_project(project_id=8afe543881644c988270ab8de48728b1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.578680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.580911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 205/418] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:27 2022] GET /identity/v3/projects/8afe543881644c988270ab8de48728b1 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:27.590390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:27.758930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79d108-8318-415a-bb8b-6c7c61cda477 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.788379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:27.817633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979ca5c-70e0-4883-be77-a5105971c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.819015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ca5c-70e0-4883-be77-a5105971c20e None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.819629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ca5c-70e0-4883-be77-a5105971c20e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.820140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ca5c-70e0-4883-be77-a5105971c20e None admin] PATH_INFO: `/v3/projects/8afe543881644c988270ab8de48728b1/groups/bcc0f4a0a8e0453bbdc44ba9d820f233/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.823301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979ca5c-70e0-4883-be77-a5105971c20e None admin] RBAC: Authorizing `identity:create_grant(project_id=8afe543881644c988270ab8de48728b1, group_id=bcc0f4a0a8e0453bbdc44ba9d820f233, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.825894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979ca5c-70e0-4883-be77-a5105971c20e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.831577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ae3b5-9add-4bf7-a7e8-f65b6fb5e9c1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:27.853800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 206/419] 10.222.0.79 () {58 vars in 1442 bytes} [Wed Jul 20 19:14:27 2022] PUT /identity/v3/projects/8afe543881644c988270ab8de48728b1/groups/bcc0f4a0a8e0453bbdc44ba9d820f233/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:27.876968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:27.878292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.878583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.878924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.937426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725addc-3d40-4448-9c84-9a17da488de9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.937768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725addc-3d40-4448-9c84-9a17da488de9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.937996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725addc-3d40-4448-9c84-9a17da488de9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.941629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 207/420] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:27.947125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:27.947417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:27.947702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:27.971868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:27.973468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:27.990052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:27.991630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:28.017072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.106343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.108774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 208/421] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:28.130473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 214/422] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:28.133803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:28.136076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:28.258596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a178-cbfe-4d0a-a525-5e532f8180d0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:28.322260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:28.323250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:28.323788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:28.323788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:28.325030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:28.326565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:28.328553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 215/423] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:28 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:28.332793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295848d-5535-4b61-872d-125d77f7e58c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:28.337240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:28.407783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.417691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:28.418498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:28.418783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:28.419059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:28.431972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:28.440738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:28.441745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:28.442557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:28.447525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 209/424] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:28.449051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:28.463323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.522864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf927c4-80c6-4183-adb8-49fd04ad7104 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:28.598030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:28.599196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:28.599658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:28.600112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:28.601857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:28.604255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:28.607807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 216/425] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:28 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:28.614657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:28.626613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db08e88-fc6c-47d3-8f72-7a4586355dbc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:28.672869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:28.675667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:28.676159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:28.676900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:28.679046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:28.679561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:28.682087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:28.719804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 210/426] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:28.726333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:28.802741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f18c-9c16-4720-9ef8-aaf325da6a0e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:28.825339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.841169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:28.842183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:28.842698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:28.843183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:28.844204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=3f00ea992c824af582c7fbffe696728b) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:28.844696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:28.846886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaf0a0-109e-498c-8cb6-343b0765a2a4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:28.854375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 217/427] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:28 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:28.889689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:28.941748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc382bd3-6deb-4032-963a-d0d31a14e207 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.002698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.003398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.003775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.004055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.100753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:29.102261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:29.132069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beafa7-2d30-4bea-97b8-54da87365d11 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.132697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beafa7-2d30-4bea-97b8-54da87365d11 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.133076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beafa7-2d30-4bea-97b8-54da87365d11 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.139368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 218/428] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:29.141149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.141545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.141894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.178296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:29.179627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:29.187229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:29.227142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 211/429] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:14:29.233918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:29.291749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 219/430] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:29.321620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:29.372488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d8435-2a18-4341-b8c7-b8ea7f7773b8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.419119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.421238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.421664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.422103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] PATH_INFO: `/v3/projects/3f00ea992c824af582c7fbffe696728b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.423898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=3f00ea992c824af582c7fbffe696728b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:29.426282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:29.428932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 212/431] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:29 2022] GET /identity/v3/projects/3f00ea992c824af582c7fbffe696728b => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:29.434203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:29.450544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb3007-537f-4eec-b909-96c79fd03c2d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.467060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:29.490273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:29.493637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.494470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.494763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.495048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.514279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:29.523237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:29.524157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:14:29.525383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:29.528233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 220/432] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:29.532840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:29.571827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166362-7a0a-4154-9b93-378e115ea6e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.611230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.613049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.613657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.614263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.615709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:29.617511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:29.619731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 213/433] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:29 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:29.625541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:29.640464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08064a06-e004-4f0a-ba44-e1b79d0d5e0a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.678393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.678981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.679263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.679537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.680284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] RBAC: Adding query filter params (name=member) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:29.680590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:29.682027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:29.692144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 221/434] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:14:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:29.697748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:29.737771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:29.758446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed15c3-5b37-4854-aae6-ead9b5d12879 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.766811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:29.798861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.801025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.801780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.802272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.803460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=3f00ea992c824af582c7fbffe696728b) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:29.804280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:29.806391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:29.809431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a6e1f-ad0b-419d-ab4a-ca0d9d3aa769 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:29.815512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 214/435] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:29 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=3f00ea992c824af582c7fbffe696728b&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:29.854743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10954c52-0e09-472f-8cc4-55aebbd23e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:29.855549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954c52-0e09-472f-8cc4-55aebbd23e4a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:29.855903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954c52-0e09-472f-8cc4-55aebbd23e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:29.856247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954c52-0e09-472f-8cc4-55aebbd23e4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:29.942649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10954c52-0e09-472f-8cc4-55aebbd23e4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:29.944042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10954c52-0e09-472f-8cc4-55aebbd23e4a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:29.983626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.063415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 222/436] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:30.069990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.183136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef5987-6bbe-4c57-9848-79341b0df805 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.204300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbb4d2-937d-4dcf-86ce-3034a1555907 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.204568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbb4d2-937d-4dcf-86ce-3034a1555907 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.204790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbb4d2-937d-4dcf-86ce-3034a1555907 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.207149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 223/437] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:30.212606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.212917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.213198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.222765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.223638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.223922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.224198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.225429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:30.226874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:30.228790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 215/438] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:30.232757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:30.233935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.234045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:30.277307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.326835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 224/439] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:30.350882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.358571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc8a6-5de0-4c44-8b39-af7561f8aefc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.396872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ea7d-614d-4b83-92f9-455c4ad219b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.397768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ea7d-614d-4b83-92f9-455c4ad219b4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.398055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ea7d-614d-4b83-92f9-455c4ad219b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.398334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ea7d-614d-4b83-92f9-455c4ad219b4 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.399522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8ea7d-614d-4b83-92f9-455c4ad219b4 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:30.401000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8ea7d-614d-4b83-92f9-455c4ad219b4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:30.403042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 216/440] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:30.404205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda6e9e-b914-48bc-a4a1-7a8748d73e45 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.404497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda6e9e-b914-48bc-a4a1-7a8748d73e45 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.404776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda6e9e-b914-48bc-a4a1-7a8748d73e45 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.406140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 217/441] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:30.408908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda6e9e-b914-48bc-a4a1-7a8748d73e45 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.422612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.458916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a4476-4794-4055-a798-00fffcfe374b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.494183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eae84-9738-4d92-9fa2-589c9c91326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.494992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eae84-9738-4d92-9fa2-589c9c91326d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.495278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eae84-9738-4d92-9fa2-589c9c91326d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.495557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eae84-9738-4d92-9fa2-589c9c91326d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.505682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563eae84-9738-4d92-9fa2-589c9c91326d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:30.512947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda6e9e-b914-48bc-a4a1-7a8748d73e45 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-563eae84-9738-4d92-9fa2-589c9c91326d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:30.513861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:30.514800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:30.515580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 225/442] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:30.516681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.516681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.516947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.523807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.534030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:30.535355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:30.552996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.553657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.553943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.554218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.555052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=cceb4e5cab5449d893b958c6130869e3, scope.project.id=c5c6b6badab543ceb85ba21c445514a4) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:30.555363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:30.556819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:30.562720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 218/443] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=cceb4e5cab5449d893b958c6130869e3 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:30.563488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.610270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 226/444] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:30.634565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.685944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664ed64-6b81-4f93-a08d-b610c1324bc8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.701747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.732965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.733881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.733881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.734074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.734839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:30.735147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:30.738322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:30.747222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 219/445] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:30.753279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.757238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73b5b-f2b1-41c1-85d1-d7d2c93f8034 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.796195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.797665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.798635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.798931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.799957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.813467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:30.823221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:30.824133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:30.824962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 227/446] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:30.830564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:30.890537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1a978-db5b-4986-9e98-869bd88a9ef7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.935517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.936467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.936998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.937414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.959765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964e8d-cd6e-4b6f-a306-0be2b33da4c4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:30.975789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:30.993621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:30.994218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:30.994501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:30.994778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:30.995529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:30.995840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:30.997244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.006038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 228/447] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:31.008795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.010287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.011541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.041705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:31.111690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dc8b6-28b0-4de9-9659-dbaee058cda5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.112283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 220/448] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:14:31.117813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.146326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.146976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.147262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.147548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.193746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:31.216332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.216668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b1fb3-5a76-46f7-b58e-a20d6ed78b4d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.217822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.246206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:31.251650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.253191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.253502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.253717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] PATH_INFO: `/v3/projects/c5c6b6badab543ceb85ba21c445514a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.254898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] RBAC: Authorizing `identity:get_project(project_id=c5c6b6badab543ceb85ba21c445514a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.256325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.258041 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 221/449] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/projects/c5c6b6badab543ceb85ba21c445514a4 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:31.263480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.319684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 229/450] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:14:31.325200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.369284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a8c43-0132-4c17-913b-d6d23215ec53 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.403574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:31.404820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.406010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.406299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.406574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] PATH_INFO: `/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.407757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] RBAC: Authorizing `identity:get_group(group_id=fc220d7c057847fdbf2c2cbd6d9216da)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.409902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.411512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 222/451] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/groups/fc220d7c057847fdbf2c2cbd6d9216da => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:31.420203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.430907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fd56a-e557-4514-a22f-f46170746d04 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.469856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.470722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.471567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.471844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] PATH_INFO: `/v3/projects/8afe543881644c988270ab8de48728b1` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.473169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] RBAC: Authorizing `identity:get_project(project_id=8afe543881644c988270ab8de48728b1)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.474624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.476330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 230/452] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/projects/8afe543881644c988270ab8de48728b1 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:31.481787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.494523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:31.528268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cebbbc-04f9-4525-b04a-7d11c2426ed6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.562248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.562903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.563183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.563459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.564594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] RBAC: Adding query filter params (group.id=fc220d7c057847fdbf2c2cbd6d9216da, role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=c5c6b6badab543ceb85ba21c445514a4) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:31.564734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.566433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.572453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 223/453] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/role_assignments?group.id=fc220d7c057847fdbf2c2cbd6d9216da&scope.project.id=c5c6b6badab543ceb85ba21c445514a4&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:31.582862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ea3bb-0134-440e-ab83-9dde57dbd409 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.624861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc7014-9db3-4804-b246-a5b24865f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.625781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc7014-9db3-4804-b246-a5b24865f220 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.626065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc7014-9db3-4804-b246-a5b24865f220 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.626337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc7014-9db3-4804-b246-a5b24865f220 None admin] PATH_INFO: `/v3/groups/bcc0f4a0a8e0453bbdc44ba9d820f233` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.627707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc7014-9db3-4804-b246-a5b24865f220 None admin] RBAC: Authorizing `identity:get_group(group_id=bcc0f4a0a8e0453bbdc44ba9d820f233)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.629243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc7014-9db3-4804-b246-a5b24865f220 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.631042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 231/454] 10.222.0.79 () {56 vars in 1180 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/groups/bcc0f4a0a8e0453bbdc44ba9d820f233 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:31.637175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:31.754996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf26b2-c0f5-4443-90bb-9d4281966627 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:31.793342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:31.798520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:31.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:31.799769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:31.800092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:31.801026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] RBAC: Adding query filter params (group.id=bcc0f4a0a8e0453bbdc44ba9d820f233, role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=8afe543881644c988270ab8de48728b1) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:31.801348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:31.802832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930262-90c3-44d8-9af2-b0356ab9597f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:31.809471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 224/455] 10.222.0.79 () {56 vars in 1376 bytes} [Wed Jul 20 19:14:31 2022] GET /identity/v3/role_assignments?group.id=bcc0f4a0a8e0453bbdc44ba9d820f233&scope.project.id=8afe543881644c988270ab8de48728b1&role.id=689534f62373409f9d02ea4acc335ab7 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:34.377134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b364a5-4144-4530-9518-bf3832a89f7e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.378158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b364a5-4144-4530-9518-bf3832a89f7e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.378698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b364a5-4144-4530-9518-bf3832a89f7e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.380537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 232/456] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:34.390275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e6354-afaf-4b30-87a3-0143452c0d6a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.390717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e6354-afaf-4b30-87a3-0143452c0d6a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.390819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e6354-afaf-4b30-87a3-0143452c0d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.421916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a05e6354-afaf-4b30-87a3-0143452c0d6a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:34.423603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a05e6354-afaf-4b30-87a3-0143452c0d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:34.518930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 225/457] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:34.568704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b364a5-4144-4530-9518-bf3832a89f7e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:34.594208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:34.742693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b364a5-4144-4530-9518-bf3832a89f7e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:34.746907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099d93-33b1-42d3-916e-219f9b719baa None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.747497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099d93-33b1-42d3-916e-219f9b719baa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.747970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc099d93-33b1-42d3-916e-219f9b719baa None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.749589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 226/458] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:34.755390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7df8-6525-437c-b76f-4f562a018de6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.755971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7df8-6525-437c-b76f-4f562a018de6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.756444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7df8-6525-437c-b76f-4f562a018de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.793925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-178c7df8-6525-437c-b76f-4f562a018de6 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:34.795486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-178c7df8-6525-437c-b76f-4f562a018de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:34.795851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:34.796845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.797300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.797718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.822554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:34.824722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b7a1c0e6-320b-4253-82d9-af0756aa3a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:34.857594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:34.858565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:34.859397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:34.859397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:34.859397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:34.859397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:34.861868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 233/459] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:34.863701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc053b5-7135-4cc7-83ce-9c10d2baddc0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.864200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc053b5-7135-4cc7-83ce-9c10d2baddc0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.864669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc053b5-7135-4cc7-83ce-9c10d2baddc0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.866844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 234/460] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:34.867791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc053b5-7135-4cc7-83ce-9c10d2baddc0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:34.908605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 227/461] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:34.910379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:34.910898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:34.911377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:34.931904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:34.933493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:34.974410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:34.998598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:35.019260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 228/462] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:35.020522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.036268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc053b5-7135-4cc7-83ce-9c10d2baddc0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.091191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.091874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.092154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.092584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.094040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:35.094511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.097336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.119391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 235/463] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:35.120742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.159528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab348fb-df48-4ade-8de4-84e39c1d7b02 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.198065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.199514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.199943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.200374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.216756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.218272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.239214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f641c4-80f3-4c12-a98b-48594626bb12 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:35.240414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:35.241310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:35.242176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:35.242176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:35.242176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:35.242176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:35.242176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 229/464] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:35.242446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.275893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:35.288783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.289613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.289909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.290178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.290780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:35.305757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.307732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c9afa2a4-a9a5-4feb-8689-d5c5251b892f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:35.332176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:35.333164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:35.334006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:35.334006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:35.334006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:35.334006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:35.335344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 236/465] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:35.340785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749f5a8-491a-4586-bc59-03dc4cc7803a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.340785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749f5a8-491a-4586-bc59-03dc4cc7803a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.340785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749f5a8-491a-4586-bc59-03dc4cc7803a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.340785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 237/466] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:35.340785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749f5a8-491a-4586-bc59-03dc4cc7803a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.374585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1409-4ebe-4a5b-82b0-0bbfb811db23 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.414406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.415490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.416088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.416385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.417291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.419107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.452920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 230/467] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:35 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:35.452920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.480106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749f5a8-491a-4586-bc59-03dc4cc7803a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.522542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.523278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.523680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.524130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.524926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:35.525243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.526699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9b14-23bf-4fe1-a30e-877ec15121e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.547296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 238/468] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:35.549052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.549511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.550043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.568870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:35.570462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:35.580818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:35.591777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd51cf-0753-4324-88f5-7ff8e71630cb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.609890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:35.647652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.648324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.648609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.648890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.649804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:35.649973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.651414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.666774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 239/469] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:35.667815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.668470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 231/470] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:35.677473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.796345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bde35f-e623-452c-8c78-8090ac4e3ee9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.825042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c316dac-0828-46ff-b979-02f26d3df338 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:35.837862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.838848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.839333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.839765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.840602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.842378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.864789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 240/471] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:35 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:35.865960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:35.873031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceef42-3926-43fa-a64c-23379765ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:35.873878 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceef42-3926-43fa-a64c-23379765ce98 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.874264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceef42-3926-43fa-a64c-23379765ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.874643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceef42-3926-43fa-a64c-23379765ce98 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.875475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cceef42-3926-43fa-a64c-23379765ce98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:35.878024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cceef42-3926-43fa-a64c-23379765ce98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:35.909452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 232/472] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:35 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:35.910736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:35.912136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4938f3-07ca-431e-83f1-c17e2d810348 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.912611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4938f3-07ca-431e-83f1-c17e2d810348 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.913028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4938f3-07ca-431e-83f1-c17e2d810348 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.914674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 233/473] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:35.922758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:35.923117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:35.923489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:35.958162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:35.959920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:35.968927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:36.003858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed007e9-d6d0-47ec-8be7-8f652f01be15 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:36.046533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:36.047391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:36.047686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:36.047959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:36.068019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:36.070374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:36.090165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:36.091072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:36.091925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:36.091925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:36.091925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:36.091925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:36.091925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 241/474] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:36.100418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:36.108347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 234/475] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:36.133049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:36.216176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:36.247643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d008c38-295c-4a22-b33e-1c11aaa749f1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:36.271734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1afc-28bd-4b78-8168-8d37240608ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:36.300045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:36.301226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:36.302039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:36.302508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:36.303736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:36.304242 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:36.305991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:36.322292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 242/476] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:36.326864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:36.336064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:36.337267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:36.337771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:36.338349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:36.339298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:36.340862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:36.349988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:14:36.349988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 19:14:36.349988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [parameters: {'id': 'f48eef0907314bc080ef582002382b85', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 19:14:36.349988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jul 20 19:14:36.369342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:14:36.370234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:14:36.371211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:14:36.372078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:14:36.373019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:14:36.373830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': 'f48eef0907314bc080ef582002382b85', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:36.374595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 19:14:36.375378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:36.379700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 235/477] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Jul 20 19:14:36 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:36.385780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:36.390680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:36.515142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140ef24-a242-4060-bdb1-1d542b7acee3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:36.557151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:36.558175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:36.559130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:36.559630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:36.560951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:36.562608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:36.572587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:36.588413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 243/478] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:36 2022] POST /identity/v3/users => generated 249 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:36.609398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a797a8-93f6-4d01-89ac-2de36b36fcb2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:36.652479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622dc1-9d20-4085-b2b3-4e3fff19cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:36.653977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622dc1-9d20-4085-b2b3-4e3fff19cd08 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:36.654457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622dc1-9d20-4085-b2b3-4e3fff19cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:36.654945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622dc1-9d20-4085-b2b3-4e3fff19cd08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:36.668477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19622dc1-9d20-4085-b2b3-4e3fff19cd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-19622dc1-9d20-4085-b2b3-4e3fff19cd08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:36.677750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:36.678652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:14:36.679431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:36.682490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 236/479] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:14:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:36.690813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:36.815347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc10e22-449d-4825-b220-d8de6c4e0da0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:36.851097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:36.851773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:36.852056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:36.852329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:36.853112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:36.853420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:36.854802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9264e-2546-4c97-8741-74cec9c72983 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:36.864457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 244/480] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:38.087085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ceb36-a5ab-4a32-8d95-fd91d8251271 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.087681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ceb36-a5ab-4a32-8d95-fd91d8251271 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.087681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ceb36-a5ab-4a32-8d95-fd91d8251271 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.089317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 237/481] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:38.094509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.094950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.095141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.119351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:38.120561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:38.183230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:38.207484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 245/482] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:38.254348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ceb36-a5ab-4a32-8d95-fd91d8251271 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:38.338709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:38.395017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ceb36-a5ab-4a32-8d95-fd91d8251271 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:38.443530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1c451-afa4-4b06-9bd2-dfb7025231c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:38.444340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c451-afa4-4b06-9bd2-dfb7025231c2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.444620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c451-afa4-4b06-9bd2-dfb7025231c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.444920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c451-afa4-4b06-9bd2-dfb7025231c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.465782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1c451-afa4-4b06-9bd2-dfb7025231c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b2d1c451-afa4-4b06-9bd2-dfb7025231c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:38.474528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:38.475402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:38.476199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 238/483] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:38.481769 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:38.520346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c699-3293-4b55-80df-6e4904c431e8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.521033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c699-3293-4b55-80df-6e4904c431e8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.521533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c699-3293-4b55-80df-6e4904c431e8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.523373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 239/484] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:38.528814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de77cd-dc91-46b3-802f-931cc2b3acf5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.529412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de77cd-dc91-46b3-802f-931cc2b3acf5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.529899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de77cd-dc91-46b3-802f-931cc2b3acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.550340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-95de77cd-dc91-46b3-802f-931cc2b3acf5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:38.551872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-95de77cd-dc91-46b3-802f-931cc2b3acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:38.613837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8b727-acc2-46a1-9d0b-0e98f517175f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:38.636140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 240/485] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:38.638034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176cc7-035d-4781-81fb-25eaf05982d6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.638332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176cc7-035d-4781-81fb-25eaf05982d6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.638615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176cc7-035d-4781-81fb-25eaf05982d6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.645353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 241/486] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:38.647012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.647303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.647579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.650617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:38.651472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.651853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.652224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.653545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:38.653957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:38.655421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:38.665029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 246/487] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:38.669369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:38.670658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:38.670778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:38.728431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:38.739782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:38.777024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 242/488] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:38.778432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:38.802710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fac3e-671c-4a73-873a-951e47e3da76 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:38.841403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:38.842207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.842530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.842808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.855267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:38.857339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:38.882042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:38.882915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:38.883797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:38.883797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:38.883797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:38.883797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:38.884415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 247/489] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:38.887887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:38.908657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffae9f3-02d0-4f83-a5b4-1f8ad5711fb8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:38.948221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:38.949172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:38.949523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:38.949872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:38.963694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:38.972786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:38.973873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:38.974712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:38.978915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 243/490] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:38.979805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.024153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.030546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.031182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09169d40-72ca-4ff7-85c1-c35fe75fd420 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.086380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.088207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.088984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.089478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.098352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d87331-b223-4654-83a1-3ede2b14af48 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.105082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:39.117171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:39.118080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:39.118876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:39.127020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 248/491] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:39.127020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.150998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.151581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.151863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.152133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.154180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:39.154180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.156198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.183729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 244/492] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:39.184763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.261752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3435a-a36b-4929-8243-4c6223fd360a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.306631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.307785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.308267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.308548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.309353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:39.309662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.311554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.319528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39e91-e472-45c1-b9e1-843740b0b1b8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.321908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.322776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 249/493] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:39.323688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.352161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.366937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.367611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.367894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.368166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.368955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:39.369262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.370634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aeb94-d531-4c71-8f30-bfb0e297a686 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.381282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 245/494] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:39.381282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc50ef-1cc5-4c74-b4d7-665c40aa3374 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.381547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc50ef-1cc5-4c74-b4d7-665c40aa3374 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.381827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc50ef-1cc5-4c74-b4d7-665c40aa3374 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.383068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 246/495] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:39.383744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc50ef-1cc5-4c74-b4d7-665c40aa3374 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.451934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c033e-27ce-49bc-9c37-239eec1e54fa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.490642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.491432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.491715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.491996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.499898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.500044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc50ef-1cc5-4c74-b4d7-665c40aa3374 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.501387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-72bc125a-5d12-494f-97bd-040f5db4a44e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:39.518542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:39.519492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:39.520387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 250/496] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:39.521490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.521490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.521609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.540446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:39.541677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:39.544476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.545311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.545596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.545868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.557469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.559022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:39.577374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:39.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:39.579045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:39.579045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:39.579045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:39.579045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:39.579045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 247/497] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:39.579428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.601397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.630469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 251/498] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:39.631319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.642332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.695837 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97967bb7-c634-4404-b26c-45ad5381df48 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.732161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.732873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.733186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.733471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.739693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f0d49-fdc2-4aeb-83d2-a1ae2e409aa8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.745074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.746527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e33cd3e7-1fbc-4bd0-8431-bc845ad81d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:39.764022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:39.765044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:39.765916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:39.765916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:39.765916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:39.765916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:39.770523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 248/499] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:39.771690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f1eb4-3dbe-4244-8c45-ad4dcf7db9f3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.772086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f1eb4-3dbe-4244-8c45-ad4dcf7db9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.772368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f1eb4-3dbe-4244-8c45-ad4dcf7db9f3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.773817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 249/500] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:39.774502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f1eb4-3dbe-4244-8c45-ad4dcf7db9f3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.774954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.775628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.775919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.776199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.777369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:39.777473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.779156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.817155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 252/501] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:39.818006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.847516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.878995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f1eb4-3dbe-4244-8c45-ad4dcf7db9f3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.886176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:39.915810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.916576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.917099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.917388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.918103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:39.918411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.919770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.922740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fae22-8111-4fed-8643-08aa1fa21274 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:39.931431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 250/502] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:39.932224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:39.962961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:39.963760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.964043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.964323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:39.974619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:39.976127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bb9bb4d0-d065-4c81-834e-f517c60fe226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:39.994061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:39.994938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:39.995839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:39.995839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:39.995839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:39.995839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:39.995839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 253/503] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:39.996746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:39.997060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:39.997340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.013933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:40.015143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:40.042121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f96dd-3bea-4319-bc65-c46906d81af4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.072172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.078211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.078870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.079152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.079424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.080298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:40.080867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.082298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.085196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 254/504] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:40.085921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.096823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 251/505] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:40.097524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.141597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.200825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfb317-ec5e-478f-9d75-ad5d8d4ca94b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.206439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc530f-40b1-4299-bdb2-ea7fc9d5ae85 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.241047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.241826 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.242112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.242385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.248836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5688741-706c-4767-a579-70001749cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.249957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5688741-706c-4767-a579-70001749cc96 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.250156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5688741-706c-4767-a579-70001749cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.250537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5688741-706c-4767-a579-70001749cc96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.260512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5688741-706c-4767-a579-70001749cc96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.262022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5688741-706c-4767-a579-70001749cc96 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e5688741-706c-4767-a579-70001749cc96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:40.278805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:40.279874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 252/506] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:40.280766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5688741-706c-4767-a579-70001749cc96 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.315145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.316868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.336113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.367831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.389182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5688741-706c-4767-a579-70001749cc96 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.420294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 255/507] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:40.421184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.424636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.425300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.425589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.425869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.426599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:40.426909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.428296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.440638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 253/508] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:40.441539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.529651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624f07-6c30-4e23-8564-95322fa74040 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.564430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef09e97-64b0-4b92-9b39-e7c6467fbfeb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.564802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1782e-464b-48df-a682-821d85447109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.565748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1782e-464b-48df-a682-821d85447109 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.566038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1782e-464b-48df-a682-821d85447109 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.566319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1782e-464b-48df-a682-821d85447109 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.573598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1782e-464b-48df-a682-821d85447109 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.575073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1782e-464b-48df-a682-821d85447109 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.577868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4df1782e-464b-48df-a682-821d85447109 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:40.591614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:40.592684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 256/509] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:40.593615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1782e-464b-48df-a682-821d85447109 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.605908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.608666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.609527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.609795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.610073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.620403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:40.629634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:40.630546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:40.631334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:40.631918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 254/510] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:40.632746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.703861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1782e-464b-48df-a682-821d85447109 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.737962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.738630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.738909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.739180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.740007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:40.740311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.741179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf74b4-15f4-4c1c-9b4f-04e93bd126e6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.741766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.759967 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 257/511] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:40.760774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.777826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.778628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.778897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.779174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.789954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.791408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:40.808453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:40.809380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:40.810228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:40.810228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:40.810228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:40.810228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:40.810228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 255/512] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:40.810502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.829879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.836610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:40.866335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c4677-394e-4fd3-b456-3284a2ed5b85 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.899899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.900563 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.900845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.901469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.901833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.903509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:40.913922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a1333-206e-4c94-b45a-7384d9899517 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:40.923256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 258/513] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:14:40 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:14:40.924997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:40.967650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:40.968586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:40.969100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:40.969519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:40.970957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:40.971476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:40.973182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.015353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 256/514] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:41.016449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.050096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f296f0-b3ef-466a-bc14-e37bf12f5ad9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.078616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.093535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.095077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.096240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.096240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.097681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:41.097879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.099718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.109308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 259/515] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:41.109944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.112919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.136592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997b0d3-e0c3-4be5-9111-06f871009ff8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.178956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.179610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.179895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.180171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.238274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09b298-82ef-44a2-8ee4-ddc08e52adbf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.278344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.279032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.279485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.279764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.280456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.280988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:41.280988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.282217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.282398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.302354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 260/516] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:41.302354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.357909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.373496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.400691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 257/517] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:41.401398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.445835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eff42-5a57-4a09-8ea2-2a764721a32c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.491140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.491947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.493696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.493696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.515087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d9fd3-1616-40d7-93aa-db50994765ba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.563483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.564739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.565261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.565544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.569465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.571160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.577139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.578565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:41.598026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:41.598935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:41.599772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:41.599772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:41.599772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:41.599772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:41.599772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 258/518] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:41.600042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.642538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.647460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.719513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 261/519] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:41.721536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.737572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c33ef9-c207-477d-95da-ea3135235538 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.790820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.792163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.792660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.793179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.805117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.807744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:41.825308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:41.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:41.827024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 259/520] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:41.827490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.836616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fa49f-2e65-457a-a195-54defba2b66c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.876395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.877552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.877890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.878172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.889258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.891373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.898950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:41.911088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:41.911977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:41.912844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:41.912844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:41.912844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:41.912844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:41.912844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 262/521] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:41.913989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:41.931056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:41.941639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacf1e6-dd56-4acf-9a1f-01f36fcab68c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:41.979139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:41.979886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:41.980165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:41.980425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:41.981216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:41.981514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:41.982889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:41.995405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 260/522] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:41.996212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.026860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9af70-6523-4445-8d7a-8f015d3aa37b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.070852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.071653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.071934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.072208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.083823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.085254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:42.103494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:42.104547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:42.105792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:42.105792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:42.105792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:42.105792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:42.105792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 263/523] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:42.105792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.118263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08477d-6e83-4204-b997-ec809e6e23e6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.159041 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.159540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.159827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.160107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.160928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:42.161246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.162693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.163171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.182754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 261/524] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:42.188635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.205039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.214855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4f87-76dc-4898-b81f-b1dff2f4c6ed None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.254705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.255362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.255648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.255929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.256656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:42.256996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.258397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.270798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 264/525] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:42.271764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.308153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139e8a2-7d6d-458b-979a-28a9771ff7f7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.346174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.347144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.347430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.347703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.356099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.357573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:42.373924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:42.375046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 262/526] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:42.375943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.392080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15019a73-f88b-4a4a-a247-b07b66fd3b8c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.403939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.429916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.430586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.430870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.431436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.431859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:42.432166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.433575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.445674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 265/527] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:42.446499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.469940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.507381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41173ec4-961d-4281-bc5e-005466ec5cfe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.545819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.546583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.546873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.547152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.548146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=9918e21a598f4668b5fde7b173b647ac) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:42.548496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.550818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.564948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 263/528] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/role_assignments?user.id=9918e21a598f4668b5fde7b173b647ac&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:42.565161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c060b33-b676-4f69-b34f-44c14b4dcafa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.565276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.605536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.606451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.606911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.607254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.615514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.617350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:42.633854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:42.634791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:42.635669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:42.640270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 266/529] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:42.641379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.685263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.701140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f66195-7fa8-4bce-8601-c482336cd821 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.741282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.742218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.742642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.743057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.744074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:42.744534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.746190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.749252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.765585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972d835-6bb0-4548-82fa-3bf093513141 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.781214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 264/530] 10.222.0.79 () {56 vars in 1139 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:42.783071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.812272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.813361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.813746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.814131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.822920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.824805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:42.840694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:42.841844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:42.842729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:42.847895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 267/531] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:42.848940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:42.905972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232a642-0a88-4a23-b688-a4c47a80094d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.945113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:42.946135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:42.946624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:42.947129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:42.948261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:42.948782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:42.950632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:42.951163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:42.961732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642d1aa-775d-43ba-955d-83f1c10e90c7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:42.966336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 265/532] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:42.968263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.006096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee38a63-db66-45a3-a3a8-1834da5cfa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.006785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee38a63-db66-45a3-a3a8-1834da5cfa5d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.007071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee38a63-db66-45a3-a3a8-1834da5cfa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.007350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee38a63-db66-45a3-a3a8-1834da5cfa5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.036798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.087783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f9b1-0d01-43e3-97db-1e3413d17781 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:43.106569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee38a63-db66-45a3-a3a8-1834da5cfa5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:43.108066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee38a63-db66-45a3-a3a8-1834da5cfa5d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:43.125971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.127652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.128257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.128716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.129719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:43.130260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:43.132000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:43.145628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 266/533] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:43.146759 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.211271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.226089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 268/534] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:43.227568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50799b8c-b6e5-4065-bf2d-4a2606b6525c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.227958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50799b8c-b6e5-4065-bf2d-4a2606b6525c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.228338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50799b8c-b6e5-4065-bf2d-4a2606b6525c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.229892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 269/535] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:43.230689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50799b8c-b6e5-4065-bf2d-4a2606b6525c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.278449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964f1cc-1152-40a1-b618-cc932d55cffe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:43.293460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.318397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.319694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.320071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.320448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.321434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=cd4e27030ec042bca0c7d301998e018e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:43.321856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:43.323489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:43.336985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 267/536] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/role_assignments?user.id=cd4e27030ec042bca0c7d301998e018e&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:43.336985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.358976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50799b8c-b6e5-4065-bf2d-4a2606b6525c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:43.403543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.404386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.404795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.405213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.406185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=69dd61877f6e41ba86d2a9bb9537afc8) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:43.406608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:43.408411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f3842-8837-4dd2-a045-6c6474b3c807 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:43.415326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 270/537] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/role_assignments?user.id=69dd61877f6e41ba86d2a9bb9537afc8&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:43.417044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.417380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.417760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.436822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:43.438317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:43.480592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c4f9f-b1a5-4deb-8158-dc349dd7c797 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:43.514207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.527464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.529025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.529337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.529715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.542502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 271/538] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:43.544271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:43.548919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:43.566014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:43.577655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:43.578698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:43.580726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:43.580726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:43.580726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:43.580726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:43.580726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 268/539] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:43.585121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.631125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.735749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd2c5-d7e1-42f4-93fd-31b6a5c94271 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:43.742265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591ef07-10d1-4986-951e-cc57570be3c3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:43.791598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.792411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.792694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.793430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.795114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:43.795770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:43.796045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:43.796317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:43.799239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:43.799574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:43.801001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:43.818895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:43.828677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:43.829650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:43.830502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 272/540] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:43.841886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.842529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 269/541] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:43.843256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:43.949287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.987855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:43.998513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558682f-97d9-4b0f-9c53-03429d109deb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.045803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.046614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.046893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.047170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.059520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.061005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.081461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b16448-979d-46b2-87d9-8d203681e751 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:44.082739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:44.083576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:44.084421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:44.085371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 270/542] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:44.090767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:44.146199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.146877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.147164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.147439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.148199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:44.148507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.149911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.171431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 273/543] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:44.177691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:44.252999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e69ce-c001-4e7a-92ed-85520b686532 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.283641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.284303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.284588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.284864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.285700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:44.286016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.287456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.300371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 271/544] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:44.306581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:44.329306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:44.376230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e983cd-34b0-49a4-9d6c-7f76a557ad8e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.393251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:44.442513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.443324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.443613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.443888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.474942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.478766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:44.503705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:44.504576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:44.505456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:44.505456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:44.505456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:44.505456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:44.512590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 274/545] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:44.525109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:44.539260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d8cee-9e91-4256-9581-4f944d537c64 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.653515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6b44-c786-4ffe-b9ee-53ab03d94751 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.669507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.670180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.670463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.670737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.671589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=3e16aed371fa4c9da4ea02563fed64c3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:44.671904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.673375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.698948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 272/546] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:44 2022] GET /identity/v3/role_assignments?user.id=3e16aed371fa4c9da4ea02563fed64c3&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:44.701070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.701762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.702052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.702329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.703077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:44.703388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.704922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b62846-9b5f-4854-8b74-38c763f5420e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.720981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 275/547] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:44.725110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:44.813416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:44.857190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55958b8-84de-478b-b8c9-30dc725c25b9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:44.918734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:44.920269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.920764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.921287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.936139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:44.938872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:44.961032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4b4d-dc51-4d48-8c09-72930b5c7465 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.961657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4b4d-dc51-4d48-8c09-72930b5c7465 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:44.961779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:44.962598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:14:44.964077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:44.965513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4b4d-dc51-4d48-8c09-72930b5c7465 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.967359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 276/549] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:14:44.967461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 273/549] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:44.971993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:44.973773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:44.974296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:44.974777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:44.997658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:44.999294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:45.029292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:45.091941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbf134-9e64-40dc-9f7a-63a90ba66859 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:45.131708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:45.132393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.132685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.133015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.134009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:45.134336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:45.135871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:45.157314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:45.159962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 274/550] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Jul 20 19:14:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:45.166037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:45.166678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 277/551] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:45.207339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:45.281059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca203e2c-fa19-4a58-9f66-b145e58727f5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:45.322196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:45.322857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.323145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.323487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.414636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:45.416822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f16f91-76df-4261-8dcd-1c32e0131a76 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:45.417588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:45.421336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:45.497617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:45.498443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.498731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.499017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.519318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:45.531129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 275/552] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:45.536421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:45.537632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:45.538642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:45.539722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:45.542092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 278/553] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:45.549960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:45.586989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:45.641303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:45.649569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62ac69-aea5-4a4b-8c3c-1fca26cc2298 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:45.693709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:45.694845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.695360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.695843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.711880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:45.713771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:45.735931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:45.737098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:45.738033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:45.738033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:45.738033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:45.738033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:45.741639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 276/554] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:45.754280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:45.784371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980c70b-17df-4d76-8f6a-bb4f879d248b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:45.840572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:45.841270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.841553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.841823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.842574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:45.842880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:45.844259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb794897-e5cf-4a34-9524-6d230896da7f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:45.854114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 279/555] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:45.855298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47987600-8a83-417b-b9e0-361ed01770bf None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.855568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47987600-8a83-417b-b9e0-361ed01770bf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.855921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47987600-8a83-417b-b9e0-361ed01770bf None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.866551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 280/556] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:45.866551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47987600-8a83-417b-b9e0-361ed01770bf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:45.927360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25136ee8-4eb0-4076-905f-cfa087f82f5d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:45.958522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:45.964629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:45.965323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.965606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.965881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.966698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:45.967003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:45.968384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cd41c-ef82-4634-a998-3cffdbf731fe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:45.987650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 277/557] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:45.987650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad0bc-6820-489d-a85c-0296373a5341 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:45.987650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad0bc-6820-489d-a85c-0296373a5341 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:45.987650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad0bc-6820-489d-a85c-0296373a5341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:45.992824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.006284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47987600-8a83-417b-b9e0-361ed01770bf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.006743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-11aad0bc-6820-489d-a85c-0296373a5341 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:46.008159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11aad0bc-6820-489d-a85c-0296373a5341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:46.042359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.043152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.043435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.043709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.056321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.057806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:46.077425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:46.078869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:46.079785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:46.079785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:46.079785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:46.079785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:46.079785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 281/558] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:46.079785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.086256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 278/559] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:46.091167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fdb77-4166-4d1c-b4af-edda519e9783 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.091464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fdb77-4166-4d1c-b4af-edda519e9783 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.091744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fdb77-4166-4d1c-b4af-edda519e9783 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.093381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 279/560] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:46.100631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fdb77-4166-4d1c-b4af-edda519e9783 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.206501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f1f3a-b8e0-4790-8234-80382e2d9c13 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.214571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fdb77-4166-4d1c-b4af-edda519e9783 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.236007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.251605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.252526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.252813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.253153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.260822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.262413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.264227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.265134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.265546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.265991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.267051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:46.267710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.270189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.272387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bd0db466-e737-467f-98f4-26e604ab3911 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:46.279653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:46.280585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:46.281537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:46.283810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 282/561] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:46.285064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.285355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.285630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.297043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 280/562] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:46.298263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.308064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:46.309349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:46.386406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 283/563] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:46.387252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.431165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd0c5-e4e3-4603-82bd-27c5ed3f9804 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.469773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.470636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.471130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.471447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.482550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:46.491401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:46.493069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:46.494303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:46.497450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 281/564] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:46.498417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.508300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758a8ea-fba1-4818-85fd-c2aebb20a811 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.524093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.548409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.549367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.549683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.549934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.550733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:46.551046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.552482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.565758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.578621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 284/565] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:46.579285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.627274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a7db-21da-4b4d-b6f6-83ef66921a6e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.665024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.666364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.666990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.667599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.678779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.681045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:46.701570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:46.702881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 282/566] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:46.704183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.709776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d908c-d0f5-4a96-984b-ee4749032308 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.756216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.756975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.757264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.757542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.769962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:46.778588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:46.779827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 285/567] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:46.780845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.823955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20600273-d853-447b-aaf3-35c1f8c8650a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.824864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.844985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:46.875224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.876337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.876708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.877314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.878175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:46.878694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.880731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.891799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 283/568] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:46.891799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:46.905358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3856b9-91d1-429d-9b5f-e82954fc87e9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:46.947686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:46.948348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:46.948637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:46.948936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:46.949777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=04be59c2cfa049629282356a9e209900) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:46.950092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:46.951524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:46.964934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 286/569] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/role_assignments?user.id=04be59c2cfa049629282356a9e209900&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:46.964934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.024819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821e40a-6acc-4cf1-9ca1-6186c87b6149 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.070203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.070898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.071362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.071642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.072506 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:47.072913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.074541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.097972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 284/570] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:47.098952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.099237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.106911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d03e8e-2f22-4213-addf-ece868aad9d1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.141240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.160975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.162035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.162329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.162611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.163413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:47.164071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.166736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.180573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 287/571] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:47.181391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.226394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a18c2c-d092-41bc-95e7-3e4b9fad12cb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.265253 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.266047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.266327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.266593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.279715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.281409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:47.302170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:47.303300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:47.308912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:47.308912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:47.308912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:47.308912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:47.308912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 285/572] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:47.308912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.320410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327150c-1015-4857-91b3-7a1b5480dd1f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.362432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.363326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.363680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.364019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.375596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.377460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d3d76e36-9b35-474a-b508-a4eef04ff306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:47.396809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:47.398026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:47.399120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:47.399120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:47.399120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:47.399120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:47.403817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 288/573] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:47.404469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.404948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a73ec-2960-47c3-a328-ed32e347323a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.405231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a73ec-2960-47c3-a328-ed32e347323a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.405512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a73ec-2960-47c3-a328-ed32e347323a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.406985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 289/574] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:47.407481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a73ec-2960-47c3-a328-ed32e347323a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.423408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.428726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406352-1886-48b7-b43a-441df27116a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.468305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.469226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.469517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.469790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.481958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.485008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:47.515677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:47.516689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:47.523212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:47.523212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:47.523212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:47.523212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:47.523212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 286/575] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:47.523212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.547009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a73ec-2960-47c3-a328-ed32e347323a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.597738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.598411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.598691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.598959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.599687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:47.599997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.605243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31778c-fd3b-49f8-9b42-c7fea37b206d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.618833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 290/576] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:47.620091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.620382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.620660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.637689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:47.638900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:47.648516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb9e1a-d4be-4f8c-8a97-c60d7bee7b1d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.684830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.685820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.685820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.686068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.689717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:47.689717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.689717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.696229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.705306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 287/577] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:47.706388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.718900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.724571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 291/578] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:47.725507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.833235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de912f3f-987c-4964-ae74-9308e880d05f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.839841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac079d-37eb-4e07-96f1-20e233366c5f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:47.873965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.874731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.875082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.875436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.875643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:47.876307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:47.876667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:47.876997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:47.877094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.877270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:47.878418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.884371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:47.885841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:47.897854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 288/579] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:47.897854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:47.900954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:47.901873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:47.903198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 292/580] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:47.904092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:47.966198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:47.986090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.026810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869bd83-0d76-43ea-8c44-cecaa7541c6d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.037396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdeaae-cbea-459c-b650-57c4b35607dc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.067299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.068098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.068627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.068627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.073874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.074672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.074961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.075238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.076033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.077657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.085981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:48.092766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:48.093719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:48.094657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:48.100330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:48.101226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 293/581] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:48.102710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.105389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 289/582] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:48.105389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.216429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3851ef-d7fa-4873-b2cf-68b6053ed9dd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.226122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac371d68-bfc1-41df-9547-e309545b8c1e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.240496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.247890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.264105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.264935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.265221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.265495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.268483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.269186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.269456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.269738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.270585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:48.270900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.272363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.274209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.275640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.295465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 290/583] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:48.295654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:48.296589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:14:48.297554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:48.298083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.298153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 294/584] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:48.298153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.422284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9a61d-0d2b-4f57-a5bf-69bf8d851a16 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.432283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59093177-6363-4c48-aca9-495b5a4c13a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.460305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.461009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.461300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.461575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.462321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:48.462651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.464011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.469323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.469979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.470284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.470563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.471350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:48.471659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.477556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 295/585] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:48.477693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.477772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.490940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 291/586] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:48.491774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.537031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.537427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.603160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb47b1-d10c-4b2d-82b4-5e5dcbcca334 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.618766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0eeed-947e-41e8-ad1b-719f862bb9f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.637233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.638377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.638821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.639099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.651440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.653226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.656721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.657508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.658463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.658824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.659683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:48.659993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.661446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:48.671481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:48.672444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:48.673937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:48.673937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:48.673937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:48.673937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:48.673937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 296/587] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:48.674222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.683997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 292/588] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:48.685180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.783785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.790920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f8c97-333f-40c0-8c8c-78a61b1252aa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.805570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:48.813221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b535-87c3-40a4-b57b-16b6057d7de5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:48.831467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.832264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.832625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.833057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.847671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.849184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.853597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:48.854503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:48.854789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:48.855068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/9918e21a598f4668b5fde7b173b647ac/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:48.867680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:48.868644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:48.869659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:48.869659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:48.869659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:48.869659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:48.874806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=9918e21a598f4668b5fde7b173b647ac, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:48.876272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:48.877313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 297/589] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:48.878505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:48.894818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 293/590] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:48 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/9918e21a598f4668b5fde7b173b647ac/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:48.895977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.010062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd3460-99f4-4ad5-a1df-c865aa0200c4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.032456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aaff51-b60a-4407-a439-298ad307c6eb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.055614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.056259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.056550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.056831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.057618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:49.057938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.059368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.073609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 298/591] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:49.074495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.084090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.089161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.090409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.090895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.091492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.105923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.107465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.111468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:49.133868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:49.134851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:49.135771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:49.135771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:49.135771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:49.135771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:49.135771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 294/592] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:49.140482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.193756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d88385-ec02-4fab-a6c5-2f8de3fd783c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.230784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.231456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.231744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.232021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.232754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:49.233178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.234561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179519c8-cea0-4662-a39a-a4c5e60d390f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.250675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 299/593] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:49.252661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a677c7-522b-49e1-a37b-ce2ee1c85458 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.253025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a677c7-522b-49e1-a37b-ce2ee1c85458 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.253372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a677c7-522b-49e1-a37b-ce2ee1c85458 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.254840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 300/594] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:49.257685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a677c7-522b-49e1-a37b-ce2ee1c85458 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.270188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9896c9f-1c56-4b72-8734-ce56e52b3244 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.308370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.309245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.309535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.309815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.318861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.320279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:49.335845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:49.336841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:49.337772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:49.339194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.342136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 295/595] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:49.342979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.381479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a677c7-522b-49e1-a37b-ce2ee1c85458 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.403936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.421521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.422197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.422485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.422760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.424097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:49.424631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.426218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200661-3387-41ef-a62f-16becf3c1e2a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.445008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 301/596] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:49.445008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.445008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.445008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.458481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241675-5618-4d0d-abad-837514fd8590 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.461638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:49.462953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:49.496679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.498663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.499187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.499699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.510005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.511717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:49.530062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:49.531053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:49.532037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 296/597] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:49.532652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.556902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 302/598] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:49.556902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.610928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.660023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63a51-0ff0-4fc1-b55b-2b82ca982729 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.661521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad782f-88f6-4652-b7ca-aee316464dfa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.679909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.702165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.702837 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.703124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.703410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.704208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:49.704602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.706635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.706756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.707444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.707731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.708005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.716725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.718565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.719664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 297/599] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:49.720753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:49.737917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:49.738855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:49.739743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 303/600] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:49.740226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.831076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77529bad-5f4e-499c-8d71-56796bf60a8a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.850303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbbfcd-e652-4dfe-83f8-11ad4566bb00 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:49.868901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:49.869808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.870485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.870773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.871080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.871889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:49.872191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:49.873668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:49.888240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:49.888827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:49.889140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:49.889422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:49.899330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 298/601] 10.222.0.79 () {56 vars in 1139 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:49.899330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.905522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:49.913907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:49.914824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:49.915671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 304/602] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:49.916361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:49.940546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.018260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f7d6f-b50a-4a6e-9efd-7dd8bf9a5c45 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.031716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976ede8-9718-44bb-8d98-8643d81f88b7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.057655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.058548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.058832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.059108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/cd4e27030ec042bca0c7d301998e018e/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.074396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=cd4e27030ec042bca0c7d301998e018e, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.075884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.080337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.081067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.081403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.081627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.082421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:50.082734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.084176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.101181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 299/603] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:49 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/cd4e27030ec042bca0c7d301998e018e/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:50.101181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.109966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 305/604] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:50.111023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.187980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.208755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.231121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173969d-81f3-4295-be88-2f04034f9795 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.262486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fdf09-f6f8-439e-9099-52042520f5c9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.295813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.297328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.297642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.299556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.317204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.318231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.318914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.319196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.319475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.319632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.320274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:50.320603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.322053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.332439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 306/605] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:50.333203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:50.356102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:50.357691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:50.360256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:50.360256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:50.360256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:50.360256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:50.360256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 300/606] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:50.360256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.466204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753d367-6041-4ceb-9ec5-047fb75a66a0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.469186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.474010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c2774-066f-4259-b1be-805e52cada0f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.505216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.506457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.506850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.507237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/69dd61877f6e41ba86d2a9bb9537afc8/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.511803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.512603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.512903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.513183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.516048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.525783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.527387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.528030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=69dd61877f6e41ba86d2a9bb9537afc8, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.529999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.547167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 307/607] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:50 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/69dd61877f6e41ba86d2a9bb9537afc8/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:50.549227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:50.555009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:50.555993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:50.557303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:50.557303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:50.557303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:50.557303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:50.557303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 301/608] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:50.571373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.679992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87760500-f710-4afb-a350-da6ba6e6ff7b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.763121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.763874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.764161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.764548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.765336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:50.765646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.767684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.796032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 308/609] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:50.801992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db692fd7-2504-4d2e-b3bf-6ffd2703e714 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:50.805009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.824205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.849566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:50.850238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:50.850595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:50.850942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:50.851880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:50.852247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:50.854075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:50.867611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 302/610] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:50.873543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:50.940565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:50.975842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fa18e-ac22-41a3-adfa-a46575ce9b46 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.005352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320ce54-6a86-4767-b070-8f816338eac2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.052316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.053471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.053764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.054043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.066476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.067350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.067637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.067919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.072112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.073916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.081209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.082675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:51.093046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:51.093977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:51.094908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 303/611] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:51.100394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:51.120124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:51.122260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:51.124344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:51.131154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 309/612] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:51.132078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.237978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:51.239633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89380-332e-4ee9-955f-6a05be188684 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.255213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:51.284916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.284916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.284916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.284916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.285334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:51.285572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.287291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b2946-9522-4539-b162-bff12a954e45 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.290686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a02a23-cc4a-4f0b-89f3-b812d41c8557 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.306864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 304/613] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:51.308339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67b093-aa3e-4b81-b147-0eca762f71c0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.308613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67b093-aa3e-4b81-b147-0eca762f71c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.308925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67b093-aa3e-4b81-b147-0eca762f71c0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.310826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 305/614] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:51.316865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67b093-aa3e-4b81-b147-0eca762f71c0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.334655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.335863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.336448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.336992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.338043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:51.338576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.340655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0332587-a825-400a-81c7-2796835ab71a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.362815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 310/615] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:51.364970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.365596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.366130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.388511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:51.390455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:51.433318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67b093-aa3e-4b81-b147-0eca762f71c0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.472408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.473413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.473585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.473838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.488988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.488988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.489711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 311/616] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:51.491078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:51.508553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:51.509537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:51.510664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:51.510664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:51.510664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:51.510664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:51.510664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 306/617] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:51.518457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.541242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:51.588458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:51.636522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40da93-8fd7-471b-afb5-c584eaaf5e71 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.665847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed366e-48e8-4f43-9aa2-f30c5865c526 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.676634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.677176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.677457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.677731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/3e16aed371fa4c9da4ea02563fed64c3/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.712638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=3e16aed371fa4c9da4ea02563fed64c3, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.714125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.722515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.723426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.723914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.724368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.725313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:51.725821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:51.727375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:51.736919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 312/618] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:51 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/3e16aed371fa4c9da4ea02563fed64c3/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:14:51.736919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.748608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 307/619] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:51.763423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.858692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aef1fa-8a1e-46f0-8da3-1850438c9c98 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:51.871881 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:51.903803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:51.904915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:51.905453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:51.905868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:51.918692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:51.920682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:51.929961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:51.931305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:51.932857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:51.935549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 313/620] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:51.942496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:51.974685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f21dcd-5b15-4599-8df8-1aa12308f817 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.035553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.036365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.036744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.037144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.045396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.047077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.063945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67d7fe-a658-47ad-9aef-2fc1a2eb7230 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:52.065801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:52.066753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:52.067625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:52.068057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 308/621] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:52.073713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.112332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.113477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.113997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.114720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.116593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:52.117128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.118700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.131372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 314/622] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:52.137121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.174393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:52.192237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032ba68-ee6e-4ab8-bbec-1814b7f3200d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.229413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.230170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.230530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.230882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.231758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:52.232128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.233658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.241854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:52.249151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 309/623] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:52.252772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccc93e-3c7f-4815-aad2-79fc603efb81 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.255441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.289964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.291124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.291594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.292052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.305007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.306859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:52.326450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:52.327448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:52.328371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:52.328371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:52.328371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:52.328371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:52.331209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 315/624] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:52.337617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.433339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:52.448219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d7f75-eb2e-4bfe-bc2f-f4a6ee63dfc3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.460107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32dc149-bef5-4c16-ab96-517a26fadc50 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.484682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.486255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.486562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.487059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/04be59c2cfa049629282356a9e209900/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.499552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.500231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.500519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.500928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.502419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:52.502738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.504137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad871bc6-c417-4176-9ac5-603887c28f28 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.505906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=04be59c2cfa049629282356a9e209900, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.507799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.517690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 316/625] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:14:52.519188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb834a-8614-4d84-a911-6885d3b43923 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.519480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb834a-8614-4d84-a911-6885d3b43923 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.519762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb834a-8614-4d84-a911-6885d3b43923 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.523118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 310/627] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:14:52 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/04be59c2cfa049629282356a9e209900/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:14:52.523954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.525138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 317/627] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:52.527375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.527929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.528328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.546831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:52.548030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:52.607139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:52.675445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f2847-2fd3-4a81-80f7-a6591d7c6898 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.729529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.730456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.730820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.731170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:52.741760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:52.743573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:52.762298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:52.763401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:52.764450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:52.765825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 311/628] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:14:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:52.772077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.790924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 318/629] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:14:52.827272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:52.911371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5d39-f678-4d69-ab23-fc639106f70b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.917447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:52.922771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:52.955929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edf1e-b964-46ac-acc8-4f5572603a94 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:52.997434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:52.998393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:52.998681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:52.998962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.011468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:53.018566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.019376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.019666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.019950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:53.020058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:53.020953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:53.021811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:53.022468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:53.022468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.022600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 319/630] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:14:53.027989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:53.031449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.061557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 312/631] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:14:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:53.065676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.200577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad38ab-0725-47fd-9714-77f5c58113b9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.214606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a85be-be84-4b3f-9cbd-c1e85e54a2ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.232637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:53.238498 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.239192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.239488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.239772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.240547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:53.240866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.242343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefeb12-46ff-4691-aab7-7043632f5f58 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:53.254354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55329820-ed36-44b9-8a24-89f62cf03b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.255026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329820-ed36-44b9-8a24-89f62cf03b84 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.255314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329820-ed36-44b9-8a24-89f62cf03b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.255596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55329820-ed36-44b9-8a24-89f62cf03b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.257298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 320/632] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:53.258871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa586-dfaf-4532-9a6f-e3f241d9b28b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.259168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa586-dfaf-4532-9a6f-e3f241d9b28b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.259451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa586-dfaf-4532-9a6f-e3f241d9b28b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.261410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 321/633] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:53.262286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aa586-dfaf-4532-9a6f-e3f241d9b28b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.297371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:53.341683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55329820-ed36-44b9-8a24-89f62cf03b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.343148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55329820-ed36-44b9-8a24-89f62cf03b84 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:53.427725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aa586-dfaf-4532-9a6f-e3f241d9b28b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.466232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 313/634] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:53.466829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.467579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39814-1341-491d-8767-babaadc73124 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.467687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.467912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.468014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39814-1341-491d-8767-babaadc73124 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.468191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.468289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39814-1341-491d-8767-babaadc73124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.482529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.483996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:53.489607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cdc39814-1341-491d-8767-babaadc73124 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:53.491032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cdc39814-1341-491d-8767-babaadc73124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:53.503415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:53.504377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:53.505326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:53.505326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:53.505326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:53.505326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:53.505326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 322/635] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:53.505756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.559721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:53.579640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:53.597674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 314/636] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:53.597889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc39814-1341-491d-8767-babaadc73124 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.633975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1f12-ff57-499b-8ce3-88d34feef4e2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.671873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.672379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.672662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.672960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.683959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.685456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:53.714059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:53.715039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:53.715969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:53.715969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:53.715969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:53.715969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:53.715969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 323/637] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:53.716766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.742664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc39814-1341-491d-8767-babaadc73124 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.784032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.785107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.785602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.786199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.787252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:53.787804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.789638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:53.804455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 315/638] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:53.805813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.826178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:53.829984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5777486-c6da-4017-acba-2cc2261aba11 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.866806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.867615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.867900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.868173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.878027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:53.879753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:53.887709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:53.888745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:53.889616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:53.894244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 324/639] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:53.895235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:53.935043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb94-ca0a-4a8a-8569-d1e1644b9378 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:53.983996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:53.985083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:53.985633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:53.986457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:53.987647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:53.988206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:53.989888 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.003913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2727b-80ae-44ef-b2f5-0a329ac86667 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.005168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 316/640] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:54.006356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.045267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.046104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.046352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.046625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.054552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.056134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:54.071141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:54.072205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:54.073201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 325/641] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:54.073817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.094454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.129044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fccc4-ce94-4874-96d5-34b7749f24ce None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.148905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.171409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.172493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.173062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.174199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.175280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:54.175819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.177649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.179925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314272f5-c2c8-4d15-af2a-2687ae949770 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.192584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 317/642] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:54.193791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.216614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.217541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.217823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.218094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.225799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.227246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:54.242623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:54.243549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:54.244525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 326/643] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:54.245708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.322167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc61c3-80af-440a-9315-811464d024ec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.339702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.365775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.366454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.366747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.367026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.367875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:54.368193 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.369659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.370830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f7de-2dfd-4671-b0f7-b8407c4ac72d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.394387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 318/644] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:54.394595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.419018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.420269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.420561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.420841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.433347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.433902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.434821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5a6fe5a9-dc73-4bdb-a8e0-014de0307f84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:54.453252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:54.454213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:54.455510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:54.455510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:54.455510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:54.455510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:54.455510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 327/645] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:54.456580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81703828-592b-4218-b978-a1ca7dbac921 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.456893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81703828-592b-4218-b978-a1ca7dbac921 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.457180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81703828-592b-4218-b978-a1ca7dbac921 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.458485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 328/646] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:54.462734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81703828-592b-4218-b978-a1ca7dbac921 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.525243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71233235-934b-41c1-8f99-478e12ff85cb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.565816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.566491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.566782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.567058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.567854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:54.568163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.569643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.581794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 319/647] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:54.581794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.595132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81703828-592b-4218-b978-a1ca7dbac921 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.631054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.638147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.638899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.639195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.639476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.705390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.713170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fe753-14ab-4a03-b99f-948a3203e8af None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.733724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.735199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.750423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:54.752115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.752432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.752676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.753695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:54.754003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:54.755886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c111239-19f9-4fa5-8e11-4c7ab94f9350 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:54.769568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 320/648] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:54.770661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:54.770947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:54.771221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:54.789464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:54.791164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:54.854295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 329/649] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:54.855145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.878368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 321/650] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:54.879994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:54.889577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.966210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0356e-6f15-45f4-8e59-47e53e46ffce None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.982327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:54.998260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e9342-3b5f-47d3-8590-cd006ce3efa7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:54.999794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.000446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.000735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.001039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.002423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=9918e21a598f4668b5fde7b173b647ac) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:55.002921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.004345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.010128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 330/651] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/role_assignments?user.id=9918e21a598f4668b5fde7b173b647ac&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:55.010896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.045560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.046387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.047022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.047022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.057177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.058712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:55.076830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:55.077925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:55.078866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 322/652] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:55.080127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.129338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.137218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43770-b021-4d9b-a2a0-d911fa953bbe None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.177103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.177911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.178207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.178485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.191138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.192702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.205231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e350-1c35-4c80-b436-47842c62620b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bf447bc9-4652-489a-a654-dd04eb8149c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:55.211541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:55.214280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 331/653] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed94ea-4324-4be6-8039-1a1fb4a9338c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed94ea-4324-4be6-8039-1a1fb4a9338c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed94ea-4324-4be6-8039-1a1fb4a9338c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.217920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 332/654] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:55.218566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed94ea-4324-4be6-8039-1a1fb4a9338c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.262621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.263835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.264299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.264738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.286996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:55.294864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:55.295865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:55.296707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 323/655] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:55.297612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.308111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.365065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed94ea-4324-4be6-8039-1a1fb4a9338c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.403986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.404668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.404979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.405272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.406106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:55.406416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.408747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf4c0-9cd7-418d-99f1-a55f1243648d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.412506 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47bedd-4000-411d-b08e-ea4c42d17526 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.427721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 333/656] 10.222.0.79 () {56 vars in 1137 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:55.429041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.429333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.429609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.435406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.449700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:55.450873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:55.457706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.458632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.459066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.460027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.460940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:55.461418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.463594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.478592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 324/657] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:55.483745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.529575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 334/658] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:55.530523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.563461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.612568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fe33e-aa10-4bf1-9ff0-961c9bf7aa2e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.656184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.657031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.657295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.657766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.658593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:55.659015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.660532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.661862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29db0c7-e115-4b82-a1e0-f0505a6b1120 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.669543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 325/659] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:55.670754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.709728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.710433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.710723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.711003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.713773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.787690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c78c7-e3d4-4709-acb8-532a75aba4d5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:55.788904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.790594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.817289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.825843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:55.826641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:55.826923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:55.827197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:55.835280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:55.836859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:55.851515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:55.853695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 326/660] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:55.855420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.912694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 335/661] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:55.913584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:55.956014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:55.974499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60eeb6-3f04-48f3-b685-4a7591b282a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.012226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.013154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.013452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.013721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.025671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:56.029260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef161fe-825b-456d-a817-7b306255f51d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:56.034651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:56.035598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:14:56.036448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:56.040819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 327/662] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:56.041551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.068919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.069666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.069924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.070195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.084397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.085879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.098590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:56.113790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:56.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:56.118878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:56.118878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:56.118878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:56.118878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:56.118878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 336/663] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:56.118878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.165508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a480f-7b06-4aaf-86b1-dd2554789cdc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.209593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.210258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.210556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.210822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.211619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:56.211931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.213408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.225858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.228466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 328/664] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:56.229925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.245601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6b7c6-4a22-4984-b059-1a1504adc893 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.286619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.287651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.287938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.288212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.299644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.301152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:56.325487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:56.326417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:56.327297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:56.327297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:56.327297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:56.327297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:56.327297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 337/665] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:56.327593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.355452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1313c-812e-436c-adb0-e8cfde783655 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.367611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.401983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.403433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.403946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.404648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.405471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:56.405794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.407204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.420529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 329/666] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:56.420529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.447277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a842f5-b1b8-434a-af22-2585cfe6f8c2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.491094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.491856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.492211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.492555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.493346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:56.493657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.495057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.502677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.510412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 338/667] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:56.511646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.538480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5672e31-dca5-43b1-a4ff-6f9e13fac75b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.574762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.575405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.575742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.575957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.576787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=cd4e27030ec042bca0c7d301998e018e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:56.577127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.578551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.592939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 330/668] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/role_assignments?user.id=cd4e27030ec042bca0c7d301998e018e&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:56.592939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.634790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.662254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140b984-9d11-4d29-a4e5-0a3397037642 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.718421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.719157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.719451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.719732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.721187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:56.721499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.722918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.732452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23926a15-8092-465b-8c9e-d6bbc0588e5e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.737686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 339/669] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:56.738606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.784414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.785260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.785551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.785941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.801147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.802615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.811959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:56.823945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:56.825147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:56.826156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:56.826156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:56.826156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:56.826156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:56.826156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 331/670] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:56.827107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.879399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a28ae9-302b-4e97-b036-9fa75037ef19 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:56.924417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:56.925316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.925602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.925988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.935061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:56.936599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:56.955340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:56.956626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 340/671] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:56.957839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:56.958647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.980645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd949e-46c7-4eed-8665-e24e385d0d44 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.028992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.030153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.030720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.031147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.040339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.042089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:57.058964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:57.060433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:57.061372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:57.063614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 332/672] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:57.064683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.079087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4a56-1b4e-4072-b4e2-9781de3c7067 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.104492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.126718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.127670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.127924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.128200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.128970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:57.129281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.131105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.147923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 341/673] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:57.149158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.192141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be42e-a207-4b5e-aafa-992a79d2a6ec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.234072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.235018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.235307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.235607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.236475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:57.236789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.238357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.258166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 333/674] 10.222.0.79 () {56 vars in 1139 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:57.259018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.263646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.300550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfcf0-44a2-40ac-8034-66e09fa684fd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.339082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.339666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.339952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.340230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.341068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:57.341382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.342840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49489b87-b157-4079-b7da-6dcf2ffd360b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.362919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 342/675] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:57.364486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eeabd0-eee1-4bb0-b509-e0afa80811b3 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.364785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eeabd0-eee1-4bb0-b509-e0afa80811b3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.365098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eeabd0-eee1-4bb0-b509-e0afa80811b3 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.366727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 343/676] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:57.367467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eeabd0-eee1-4bb0-b509-e0afa80811b3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.391726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830832d-8962-4d15-97cc-7fe9b0a4c422 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.412948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.432691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.433923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.434218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.434498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.442576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.444052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:57.461488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:57.462537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:14:57.463547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:57.467300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 334/677] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:57.468154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.478964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eeabd0-eee1-4bb0-b509-e0afa80811b3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.520238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.520925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.521216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.521495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.542167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.586270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497012bb-1b43-4652-aa70-412e72f22e62 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.603483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.605011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.631402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.632372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.632852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.633167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.634015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=69dd61877f6e41ba86d2a9bb9537afc8) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:57.634330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.635776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b7f05-593d-4c06-a958-7b9ded929fce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.649503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 335/678] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/role_assignments?user.id=69dd61877f6e41ba86d2a9bb9537afc8&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:57.649503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.649503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.649503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.667879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:57.669330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:57.669886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.753921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 344/679] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:57.755002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.760272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 336/680] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:57.772705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.858503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.886367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d2b0-e3d4-4803-a4fa-613103db2cf9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.918296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adc73-2622-4ed8-969c-2c3b90cf5f6c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.932962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.933718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.933997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.934453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.935709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:57.936147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.938016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.954062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.959213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.960315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.960772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.961282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.969453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 345/681] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:57.971449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.976280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.977812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:57.996997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:57.997970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:57.998851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:57.998851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:57.998851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:57.998851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:58.004921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 337/682] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:58.004921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.117326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3506-2b5b-48e5-9a35-e439d36d4646 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.134063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a81f89-f8ab-4cad-9e10-20bf116c4557 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.146990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.170717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.171528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.171818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.172100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.174164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.174903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.175264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.175617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.185466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:14:58.195900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:58.196863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:58.197727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 346/683] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:58.198635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.264607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.266789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.272830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.323261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171c33-0ba2-4895-8474-7330ee9043c6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.365428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.366107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.366395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.366678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.367527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:58.367788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.369268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.385243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 338/684] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:14:58.386498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.388052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 347/685] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:58.389009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.400195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.499460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a195a-0e0a-41bb-a68c-c66f91c2564d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.500213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55890fca-c6e8-4a59-a127-5c019d9490d3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.518631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.542335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.543899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.544232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.544464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.550516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.551523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.552050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.552543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.553327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.553795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:58.554320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.554786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.556301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:58.571150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:58.572103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:58.573497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:58.574118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 339/687] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:58.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.576596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 348/687] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:58.577672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.655217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.688373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73f989-1117-4fb4-b6cf-249ab0ecd7b1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.692306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a725a3d-5485-45a9-97db-48547b5f4018 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.732754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.733699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.734014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.734254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.736576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.737694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.738201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.738684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.747248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.748808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.751584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.753317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:58.768448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:58.769610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:58.770509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:58.770509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:58.770509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:58.770509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:58.774387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 349/688] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:58.775224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:58.776129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:58.777047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:58.777047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:58.777047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:58.777047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:58.777264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.779592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 340/689] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:58.780690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.808702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.888899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331880b-659a-4e1b-8a12-66b8a4bb14a6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.896903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb97552-d973-4eaf-86eb-ad305f9a3d7a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:58.923034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:58.926742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.927427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.927713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.928000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.928811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:58.929218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.930680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.937040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:58.938082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.939080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.939585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.940597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:58.941234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:58.942677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54780349-53a7-4c57-8f9e-e420517ee6ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:58.951398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 350/690] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:58.951398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:58.962545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 341/691] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:58.964041 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033eb039-650a-4217-a202-d8d832c6ca43 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:58.964330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033eb039-650a-4217-a202-d8d832c6ca43 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:58.964607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033eb039-650a-4217-a202-d8d832c6ca43 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:58.966044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 342/692] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:58.970348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033eb039-650a-4217-a202-d8d832c6ca43 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.079032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:59.098907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a9075-91c1-448c-83f9-cf6127d27371 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.101357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033eb039-650a-4217-a202-d8d832c6ca43 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.140256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.140927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.141212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.141492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.142340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=3e16aed371fa4c9da4ea02563fed64c3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:59.142648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.143070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.143896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.144078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.144473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.144950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.145926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:59.146382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.148010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cba3d3-b1e0-4051-9e8a-8f5e847641ed None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.150313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 343/693] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/role_assignments?user.id=3e16aed371fa4c9da4ea02563fed64c3&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:59.151078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.179072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 351/694] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:59.180617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.180943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.181236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.202101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:59.203370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:59.224680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:59.302604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f2532-3981-4d7a-8af6-ec4e458ca8f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.313439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 352/695] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:14:59.315312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.363828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.364634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.364954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.365236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.376526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.379090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:14:59.396284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:59.397308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:14:59.398231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:59.401768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 344/696] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:59.401888 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.440669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:59.476201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0808ad-5cf4-49e1-b216-306efdb0c897 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.525320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.526135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.526418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.526694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.536902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.538378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.544346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a2243-599d-484a-92be-142f9d659647 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.557029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:14:59.557592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:59.558470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:14:59.559780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 353/697] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:59.560265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.596848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.598179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.598653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.599133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.618789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.621757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:14:59.656867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:14:59.658983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:14:59.660453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:14:59.660453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:14:59.660453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:14:59.660453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:14:59.660453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 345/698] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:14:59.660957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.702151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b16b02-aab6-4b76-ae40-16fc47315d66 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.755089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.756331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.756736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.757132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.758142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:59.758646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.760713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.774480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:59.793379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 354/699] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:59.797557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.808300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710c61a-6b95-4a7c-a96b-d0bf0adc6f72 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:59.856972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:59.857980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.858431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.858766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.859727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:14:59.860240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:59.862972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:59.881199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 346/700] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:14:59.882446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:59.912717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:59.951653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e642c3-dcd4-47a9-b5e4-8f148cbda082 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.016278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929461-f11b-44b1-8243-33fa3be8ebe3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.027216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.028169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.028457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.028736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.029510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:00.029822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.031220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.052456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 355/701] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:00.053374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.076786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f111ed-8705-4685-ae91-daa612d1a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.079792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f111ed-8705-4685-ae91-daa612d1a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.080224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f111ed-8705-4685-ae91-daa612d1a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.080653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f111ed-8705-4685-ae91-daa612d1a0e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.131529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.191813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8a4db-5a04-4603-bf01-8627492ea27c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.199551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f111ed-8705-4685-ae91-daa612d1a0e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.202683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f111ed-8705-4685-ae91-daa612d1a0e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.236417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.237588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.237798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.237883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.238152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.239100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=04be59c2cfa049629282356a9e209900) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:00.239902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.241439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911522a3-2959-45c3-bc5f-78257e23191c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.256930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 356/702] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:00 2022] GET /identity/v3/role_assignments?user.id=04be59c2cfa049629282356a9e209900&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:00.256930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ce32-8f02-4975-af21-4cf61bf33e4e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.256930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ce32-8f02-4975-af21-4cf61bf33e4e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.256930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ce32-8f02-4975-af21-4cf61bf33e4e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.259602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 357/703] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:00.261667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030ce32-8f02-4975-af21-4cf61bf33e4e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.395439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 347/704] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:15:00.397141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.397451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.397769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.417347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:00.418594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:00.453876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030ce32-8f02-4975-af21-4cf61bf33e4e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.492255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.513622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 348/705] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:15:00.514484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.523362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.524050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.524340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.524620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.525380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.527291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.556379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 358/706] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:15:00 2022] POST /identity/v3/users => generated 254 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:00.562685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.612026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.695479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f1e1d-e9d9-4b96-83b6-6b56089dbf28 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.718196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a14d5a-2e7f-43b1-bdcc-ce6c5ddaec9d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.740521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.741381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.741663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.741937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.765481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.768328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.772758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.774440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.774935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.775420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.776299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.779251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:00.797559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:00.799419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:00.800305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:00.800305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:00.800305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:00.800305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:00.800548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:00.801458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:00.802311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 359/707] 10.222.0.79 () {56 vars in 1129 bytes} [Wed Jul 20 19:15:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:00.805694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 349/708] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:00.809040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.814394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.865289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.945504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155361-a862-4c23-82e2-d4bd9cd74f6e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.949914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d020f84-a20a-402f-9d8a-7eaff7bba086 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.951500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.989935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.991352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.991896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.992392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.993517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:00.994034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.995630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.996288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.997666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.997666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.997666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.998265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:00.998576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:01.000497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:01.004046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 360/709] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Jul 20 19:15:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:01.009983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:01.017562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 350/710] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:01.023384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:01.150792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9febb33-583d-48c3-ac28-0fe227ff72c2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:01.153997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:01.170803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d9591-0493-4bb4-865a-d5f43b645e98 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:01.192640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:01.193405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.193680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.193975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.194692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:01.195006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:01.197972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:01.203650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 361/711] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Jul 20 19:15:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:01.254836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:01.255922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.256362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.256793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.273874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:01.275710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:01.297687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:01.299453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:01.300533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:01.302507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 351/712] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:01.308632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:01.443833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623124c9-b177-4b85-a4c6-ffcca651706f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:01.485220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:01.485932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.486218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.486502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.487309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:01.487624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:01.489103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e366-6633-41e1-a1ae-9eb576b71791 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:01.502228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 362/713] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:01.507827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:01.536092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:01.550883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e603945-6295-4b04-b623-0f625a4d9302 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.551223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e603945-6295-4b04-b623-0f625a4d9302 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.551446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e603945-6295-4b04-b623-0f625a4d9302 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.553100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 363/714] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:01.558101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.558387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.558661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.578280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:01.579645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:01.605204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:01.625578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e501b-08bf-45ea-b112-dc4324a3bf48 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:01.655075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 364/715] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:15:01.667415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:01.668272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.668666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.669091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.670047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=9918e21a598f4668b5fde7b173b647ac) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:01.670473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:01.672020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:01.677159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:01.679388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 352/716] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:01 2022] GET /identity/v3/role_assignments?user.id=9918e21a598f4668b5fde7b173b647ac&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:01.809199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6e3b7-15d7-4e12-91fa-0c5f9b9d90e7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:01.847003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:01.847822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:01.848105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:01.848379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:01.849074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:01.850483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:01.863970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:01.865067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:01.866393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 365/717] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:01 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:01.871444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:01.990204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:02.006817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cbea4-0e91-4a60-89c5-fbb74600b2e5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:02.042044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:02.042710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:02.042990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:02.043268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:02.044141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:02.044450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:02.045839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34104aa-2feb-481a-a158-11878fb719e5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:02.051990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 353/718] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:02.058140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:02.089411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:02.173565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90cdd8-8fed-4ae6-af41-fc88fea844d3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:02.209856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:02.210529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:02.210811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:02.211085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:02.211799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:02.212104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:02.213508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d322de-d994-4cdf-bb59-52755f658bb5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:02.218649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 366/719] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:03.127015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8413c-e2e9-45ab-a825-96506c6f2f61 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.127539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8413c-e2e9-45ab-a825-96506c6f2f61 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.127618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8413c-e2e9-45ab-a825-96506c6f2f61 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.129221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 354/720] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:03.134610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.135036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.135261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.158639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:03.159845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:03.238595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 367/721] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:15:03.277932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8413c-e2e9-45ab-a825-96506c6f2f61 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:03.416994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8413c-e2e9-45ab-a825-96506c6f2f61 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:03.458185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd4275-6857-4d4a-bcf8-a230be53ee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:03.458983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd4275-6857-4d4a-bcf8-a230be53ee45 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.459264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd4275-6857-4d4a-bcf8-a230be53ee45 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.459537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd4275-6857-4d4a-bcf8-a230be53ee45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.472204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd4275-6857-4d4a-bcf8-a230be53ee45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-06dd4275-6857-4d4a-bcf8-a230be53ee45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:03.480110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:03.481165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:03.482011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 355/722] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:03.490476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:03.518963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.526671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f214d-b6de-4cdf-806d-712a959ae203 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.526999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f214d-b6de-4cdf-806d-712a959ae203 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.527290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f214d-b6de-4cdf-806d-712a959ae203 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.529069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 356/723] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:03.535281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.535851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.536333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.555302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:03.556991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:03.560393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.659992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9888b-1ee8-49cc-b677-8f90996f3ff5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:03.664795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 357/724] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:15:03.705782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:03.706617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.707003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.707419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.708328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:03.708735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:03.710387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:03.718434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:03.721418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 368/725] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:03.732825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:03.825554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.862096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08807-231c-4bb9-a622-6291af7ab3fc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:03.878494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa25708-a535-4c57-a25f-0f4a9eb408d1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:03.899805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:03.900603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.900903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.901187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.914180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:03.915757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:03.919312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:03.920189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.920540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.920872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.930861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.934958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ab310217-3862-44dc-9534-9dc05e79bdc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:03.937400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:03.939029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:03.940009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:03.940009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:03.940009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:03.940009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:03.942550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 369/726] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:03.944036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e0b3-370e-45c6-8809-464a7dee5720 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.944618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e0b3-370e-45c6-8809-464a7dee5720 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.945100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e0b3-370e-45c6-8809-464a7dee5720 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:03.945221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:03.946139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 358/727] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:03.951949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:03.956517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 370/728] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:03.957620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405e0b3-370e-45c6-8809-464a7dee5720 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.064485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae27f-775b-4e6e-b881-5a9840b09ba9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.075363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405e0b3-370e-45c6-8809-464a7dee5720 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.102368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.103313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.103763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.104248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.105198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:04.105691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.107266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df76b83-b3db-4874-aa20-470ba8d52b93 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.107386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:04.116925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.117662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.117946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.118217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.118975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:04.119280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.120650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.123980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 359/729] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:04.125625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.126826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.127339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.134873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 371/730] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:04.134873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.150772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:04.152330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:04.231535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:04.241017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd80c6-171a-4bde-a154-1115df931e88 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.253006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 360/731] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:15:04.254470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.292941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.292941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.292941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.292941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.295523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.297472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:04.314536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:04.317391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 372/732] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:04.318359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.412769 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:04.430981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57ed3-201d-4969-bfed-058be44d4e08 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.475992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.476991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.477273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.477590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.482045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e786-781d-4114-89d6-1b3cb888c796 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.490742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.492320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:04.512232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:04.513223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:04.514564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:04.514564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:04.514564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:04.514564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:04.518808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 361/733] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:04.519717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.528437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.529544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.530036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.531054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.532132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.535536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.565418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 373/734] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:04 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:04.565418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.586887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:04.702551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d90a34-43a5-4ecf-aad1-043f6fc5ce40 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.752236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e2146-45cc-4af0-85ef-bb3abf2a24c6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.758898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.759583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.759868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.760145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.761025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:04.761340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.762796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073962c-c383-4d44-b0cb-208adf63a2be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.763797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:04.787436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 362/735] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Jul 20 19:15:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:04.789063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02a8b-a120-4880-9562-cd3b49cae90b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.789490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02a8b-a120-4880-9562-cd3b49cae90b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.789894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae02a8b-a120-4880-9562-cd3b49cae90b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.791272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 363/736] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:04.793516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02a8b-a120-4880-9562-cd3b49cae90b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.809621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.810408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.810690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.811074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.811808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:04.812110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:04.813777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfa552-21f2-4a7c-9888-4b61a3e6a8ad None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:04.826575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 374/737] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:04.828112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1e942-f1a3-4c56-b788-4aa516ac8eb4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.828631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1e942-f1a3-4c56-b788-4aa516ac8eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.828749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1e942-f1a3-4c56-b788-4aa516ac8eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.849626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f5e1e942-f1a3-4c56-b788-4aa516ac8eb4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:04.850828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f5e1e942-f1a3-4c56-b788-4aa516ac8eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:04.904394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:04.914115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae02a8b-a120-4880-9562-cd3b49cae90b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:04.936185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 375/738] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:15:04.936185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ceab0-7d06-40fc-b830-b50fbcd8deae None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.936185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ceab0-7d06-40fc-b830-b50fbcd8deae None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.936185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ceab0-7d06-40fc-b830-b50fbcd8deae None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:04.936185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 376/739] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:04.936185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ceab0-7d06-40fc-b830-b50fbcd8deae None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:04.967596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:04.968257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:04.968787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:04.969123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.042282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.047335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ceab0-7d06-40fc-b830-b50fbcd8deae None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.052269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.053784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.082708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.083528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.083808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.084079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.091624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.093092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d43f1e61-d39e-405d-81a5-221335024fc5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:05.108690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:05.109723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:05.111238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 377/740] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:15:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:05.111828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.111895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.112605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.129204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:05.130392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:05.169124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.183346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 364/741] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:15:05.183346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.213231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 378/742] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:15:05.214246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.287510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e764e-4242-4ff6-bb98-bb42a6584854 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.313591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.327390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62869d1-ba68-484d-a2be-39562160af75 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.329986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.331064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.331348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.331628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.345447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:05.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:05.355179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:05.356063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 365/743] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:05.356766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.379679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.380634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.381090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.381582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.383162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:05.383722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.386274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.410270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 379/744] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Jul 20 19:15:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:05.410270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.431933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.488370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae7b74-5567-4a06-b9e4-1adc924858bb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.530926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.532013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.533151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.533878 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.540971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c90e2-948f-4453-811a-323464ae8f71 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.553374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.554859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:05.574277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:05.575224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:05.576098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:05.576098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:05.576098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:05.576098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:05.576098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 366/745] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:05.577002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.597636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.598355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.598707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.598981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.599719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.601140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.615995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 380/746] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:05 2022] POST /identity/v3/services => generated 240 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:05.616843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.628072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.749408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cc49-3b6a-4c19-a155-bbf142219aec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.762397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.801634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f14ad-0d5e-40e9-8cbc-ac1c3edb5eca None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:05.810619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.811303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.811590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.811866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.812632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:05.815131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.815131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.824790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 367/747] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:05.825588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:05.861165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:05.861829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:05.862115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:05.862393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:05.951593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:05.953114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:05.955882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:05.963498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24c8c1-fee4-4db1-b1f7-35c942170334 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.018395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.019442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.019887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.020531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.021913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:06.022381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.023927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.040915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 368/748] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:06.042060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.083345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.101947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 381/749] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:06.103030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.191418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397844f-29dd-45a0-873a-c083ff10ad29 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.229796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.230920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.231309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.232068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.242349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc674d3-3a4d-4db1-bc4c-bfd9c5ab4d2c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.245160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.248103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.250445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:06.276706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:06.278019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:06.279091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:06.279091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:06.279091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:06.279091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:06.283316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 369/750] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:06.284323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.303531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.304593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.305319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.305439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.313867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.316107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:06.332783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:06.333984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:06.335122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:06.337097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 382/751] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:06.338221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.384697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.406659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852e137-6ba7-47bf-a4b0-0ea9f78a3dcd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.444023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.445176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.445642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.446125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.458138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.460090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.463516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfddd9-2fdb-45ed-b751-568a7e740955 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:06.479680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:06.480660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:06.481571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:06.481571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:06.481571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:06.481571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:06.483831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 370/752] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:06.484901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.509360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.509827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.510112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.510389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.511134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:06.511445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.512919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.533883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 383/753] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:06.534983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.563475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.602290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afa1d4-0c23-4598-b62e-d6a374fb0a02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.638294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.639329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.639839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.640312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.641350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:06.642513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.645078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.647171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.656171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067f1ae-aa97-4eba-bf6e-ee5b984aa54d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.660662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 371/754] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:06.661799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.697035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.697779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.698067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.698345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.699120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:06.699433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.701429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.720375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 384/755] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:06.721927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.776748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bce6c-153b-45b1-880f-ef3f1e2d456c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.814703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.815494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.815780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.816073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.824226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.824431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.825788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:06.841740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:06.842751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:06.843702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 372/756] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:06.844390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.856600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbf61-49f8-456c-82e3-eb174a14f0e6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.900497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.901227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.901509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.901790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.902919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=04be59c2cfa049629282356a9e209900) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:06.903233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.904715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ed11-f3d5-422e-8765-3dc340db01ed None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.918402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.924317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 385/757] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/role_assignments?user.id=04be59c2cfa049629282356a9e209900&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:06.924317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985083f3-ff09-423c-a5fc-2953aadaf702 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.924317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985083f3-ff09-423c-a5fc-2953aadaf702 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.924317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985083f3-ff09-423c-a5fc-2953aadaf702 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.930173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 386/758] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:06.931168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985083f3-ff09-423c-a5fc-2953aadaf702 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.992704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840ffa-a2bb-4322-ab69-8f26b5aa2000 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.052044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.053736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.054272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.054969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.065395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.067728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.080931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985083f3-ff09-423c-a5fc-2953aadaf702 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-95db1e44-8577-4cdf-a92f-8ec2c500c023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:07.087502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:07.088391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:07.089341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:07.097008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 373/759] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:07.097008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.097008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.097008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.114422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:07.115854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:07.123349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.124039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.124437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.124726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.125650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:07.126031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.127588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.152975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 387/760] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:07.153871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.162308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:07.203436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 374/761] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:07.208905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.259331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:07.300325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a81bc-fcb4-4f2e-a24a-4340b21d2342 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.338810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.339481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.339658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb04a3-1356-4bbe-9932-5efd7aef0d02 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.339840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.340030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.340828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:07.341190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.342632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.357578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 388/762] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:07.358457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.387707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.388430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.388713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.389038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.404016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.406748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:07.432326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:07.433540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:07.434415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:07.434415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:07.434415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:07.434415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:07.437529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 375/763] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:07.439797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.443680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:07.487308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b3e8-123c-4738-b1a7-33a321e42643 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.530917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.533199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.533832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.534405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.535267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.537045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.548054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:07.554264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8544e-b1bb-4347-bd75-7475d9c02fab None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:07.555873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:07.556901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:07.557742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:07.560952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 389/764] 10.222.0.79 () {56 vars in 1129 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:07.562046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.592641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.593309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.593649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.593928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.595427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=f9fd37a822f04be2ab327efb57f59b0c) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:07.595747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.597427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.609078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 376/765] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/role_assignments?user.id=f9fd37a822f04be2ab327efb57f59b0c&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:07.610108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.682716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07e1b0-f31a-4e8c-b42d-cd3fe132c9c2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.715601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:07.731530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.733591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.734852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.735589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.737638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:07.738526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.740370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.762852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 390/766] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:07.769021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.790920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a6b57-bd8b-4316-9269-ab1b23b6f0ba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.834721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.836150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.836656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.837425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:07.838979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:07.839601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:07.841280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:07.850268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 377/767] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:07.856433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:07.896325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:07.948108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80361ca4-b6da-4d5c-97ce-6aca6e3e4e0a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.987584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a5840-706e-4b0b-a303-e7930ebe62e9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:07.995124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:07.996778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:07.997295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:07.997765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.008291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.010636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.027932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2d45c10c-1769-4908-9313-226789603b07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:08.028329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:08.029230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:08.030108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 391/768] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:08.031233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8f5f7-9226-4dd2-ab77-bb776fe87017 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.031516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8f5f7-9226-4dd2-ab77-bb776fe87017 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.031787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8f5f7-9226-4dd2-ab77-bb776fe87017 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.032941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.033441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.033907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.034872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:08.035375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.037199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204eeb88-d266-422c-8512-7f0eb6b15da0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.038267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 392/769] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:08.039095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8f5f7-9226-4dd2-ab77-bb776fe87017 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.046211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 378/770] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Jul 20 19:15:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:08.053038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.053971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.054433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.076993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:08.078485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:08.096981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:08.156162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8f5f7-9226-4dd2-ab77-bb776fe87017 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.170162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 379/771] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:08.171410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.191494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:08.195492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.196261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.196620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.196999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.197946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:08.198296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.199785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.213275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 393/772] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:08.214211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.284186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361879c-8c25-455e-a881-a0b7c7d39989 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.321674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.322703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.323177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.323628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.324602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] RBAC: Adding query filter params (interface=public, service_id=652c8e8ec6b94865879aa668c55619dc, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:08.325126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.325247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15110811-ccfd-49da-93ce-7fb395ed990f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.327258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.336108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:08.342049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 380/773] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/endpoints?service_id=652c8e8ec6b94865879aa668c55619dc&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:08.343566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.364479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.365685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.366047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.366396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] PATH_INFO: `/v3/services/network` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.367141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.368778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:08.385124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:08.386201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:08.387044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 394/774] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:08.387879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.443996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:08.467529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d4f5-bc22-42ef-80e6-0d818f8a5910 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.503664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1bb22-5469-49c0-8e60-6e186b3de888 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.506666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.507798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.508239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.508674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/9918e21a598f4668b5fde7b173b647ac/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.511108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=9918e21a598f4668b5fde7b173b647ac, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.512803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.530273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 381/775] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:08 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/9918e21a598f4668b5fde7b173b647ac/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:15:08.531434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.557428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.558278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.558686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.559066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.559927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.561460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.575702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 395/776] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:08.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6538a-8c80-43df-986c-9b9706f04a31 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.712139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.713500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.714032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.714487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.715466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:08.716065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.717757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.724941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 382/777] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:08.730495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.789412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:08.868449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6520b6-4afd-48fa-a658-a76559927f98 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:08.905380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:08.906054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:08.906339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:08.906613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:08.907954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:08.908260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:08.909673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:08.916410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 396/778] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:08.921892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:08.931557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:09.037370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b002-386d-400f-8eb1-2d34e36095eb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:09.082548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:09.083235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.083500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.083775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.084571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] RBAC: Adding query filter params (interface=public, service_id=3f556fdd486d4e96add28fcd17426d1e, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:09.084895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:09.086261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06fdec-9393-407f-a00a-212b3d8fc4c4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:09.092088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 383/779] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:08 2022] GET /identity/v3/endpoints?service_id=3f556fdd486d4e96add28fcd17426d1e&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:09.097746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:09.210241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:09.219121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6686e-c8fc-4612-af02-3d9dc5fc66c7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:09.264407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccfb6b-7383-4031-b81b-947bdef965e5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.264749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccfb6b-7383-4031-b81b-947bdef965e5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.264986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccfb6b-7383-4031-b81b-947bdef965e5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.266541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 384/780] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:09.267045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:09.267709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.267987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.268256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.268985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:09.270369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:09.276403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 397/781] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:09 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:09.279811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.280403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.280952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.300277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:09.302306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:09.389318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:09.393837 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 385/782] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:09.437057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:09.557313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f1e8-9228-4287-a085-1f9074bdb716 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:09.595654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:09.600900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.600900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.600900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.612739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:09.621319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:09.622384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:09.623209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 398/783] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:09.629404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:09.809259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cb215-5bd0-4416-ac1e-bebf6f3b99eb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:09.862722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:09.863404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.863688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.863967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.864721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:09.865066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:09.866462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fdffd-ccd7-459b-83ad-5d78f111064b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:09.879954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 386/784] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:09.885594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:09.944680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:09.972472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f61161-23e5-4c18-8e38-1dbe4809fcf7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.972765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f61161-23e5-4c18-8e38-1dbe4809fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.973090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f61161-23e5-4c18-8e38-1dbe4809fcf7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.974810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 387/785] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:09.980194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219947f8-ef37-4159-a49b-1b462678522f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.980482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219947f8-ef37-4159-a49b-1b462678522f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.980767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219947f8-ef37-4159-a49b-1b462678522f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.998217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-219947f8-ef37-4159-a49b-1b462678522f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:09.999417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-219947f8-ef37-4159-a49b-1b462678522f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:10.082761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.093504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d14be-51d4-4086-ab21-beaed42df410 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.101232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 388/786] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:10.120806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219947f8-ef37-4159-a49b-1b462678522f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.155582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.156441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.156730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.157569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:10.170140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:10.171669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:10.190741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:10.191653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:10.192557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:10.192557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:10.192557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:10.192557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:10.196773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 399/787] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:10.198019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.281252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.307782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae940b7-5839-4da2-b2b1-2b698fec6352 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.345919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.346498 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.346789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.347067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:10.347796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:10.348106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:10.349616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:10.367510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 400/788] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:10.369719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.372121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219947f8-ef37-4159-a49b-1b462678522f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.473651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.474835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.475373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.475869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:10.488905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086f099-3ffc-4ad0-a625-88672ebc5736 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.495001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:10.515535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:10.516673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:15:10.517774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:10.518939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.519745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.520029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.520308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:10.521389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 389/789] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:10.527093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.528219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:10.529810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:10.538395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:10.555755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:10.556742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:15:10.557812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:10.559367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 401/790] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:10.565021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.631427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.684735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e76bf-58a4-4cdf-a1ec-b90ff295239a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.723397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.724079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.724361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.724637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:10.725502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:10.725812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:10.727808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:10.750436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 402/791] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:10.754905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38ba8c-72e2-41c1-9029-4bd24adcf95a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.757151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.810869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.811547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.812437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.812721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:10.814248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:10.814559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:10.816087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:10.826061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 390/792] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:10.832239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:10.879565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.944350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3ea1-cdc0-443c-bf20-8d838e18d80f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.953479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223252-9743-4a85-98ee-b6e0b829d2b4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:10.970140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.993310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:10.994126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:10.994408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:10.994684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.007547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.009046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.009610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.010434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.010719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.010992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:11.028394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:11.029378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:11.030273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:11.030273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:11.030273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:11.030273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:11.030273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 391/793] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:11.030549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.034601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.035875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:11.053925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:11.056024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:11.058159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:11.058159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:11.058159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:11.058159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:11.064958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 403/794] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:11.064958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.159036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b4931-5ac1-490c-b2f0-4286a135b81c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.212312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.213022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.213307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.213583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.214315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:11.214621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.216185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869924f7-e7ef-4b41-8343-e2dfd04900b6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.230301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 392/795] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:11.231589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc8230-11ac-4f75-8fdf-436b0722031b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.231874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc8230-11ac-4f75-8fdf-436b0722031b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.232153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc8230-11ac-4f75-8fdf-436b0722031b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.232319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa714b-a841-4d8a-bd14-28b41dbe1c83 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.238057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 393/796] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:11.238884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc8230-11ac-4f75-8fdf-436b0722031b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.239951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:11.271661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:11.274576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.276202 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.276743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.277223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.278368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:11.278829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.280419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2827b4-2baa-44a5-b371-07f85afa53da None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.294266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 404/797] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:11.296034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.296549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.297008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.315896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:11.317502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:11.358661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc8230-11ac-4f75-8fdf-436b0722031b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.399464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.400127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.400274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 405/798] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:11.400368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.400602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.401622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.409087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.410883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:11.429945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:11.431194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:11.432163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 394/799] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:11.440426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.531567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:11.534266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:11.541332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f0d03-e8a2-4273-b28e-074154e782cf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.550868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd7dd3-3021-49ca-8f6b-d42f912be4b4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.579328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.580109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.580399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.580674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.588530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.590020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.590160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.590702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.590992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.591268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.592114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:11.592428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.593887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a9d01-c87a-4a7a-8df0-58d295ce74de None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:11.605798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:11.606795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:11.607683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:11.609051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 406/800] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:11.609864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.613917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 395/801] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:11.615585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1054275-d8d7-4729-9a86-c455b69d3c26 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.616077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1054275-d8d7-4729-9a86-c455b69d3c26 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.616553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1054275-d8d7-4729-9a86-c455b69d3c26 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.618225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 396/802] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:11.619130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1054275-d8d7-4729-9a86-c455b69d3c26 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.726728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4fe6e-ccd9-422d-a1d6-e9534276d91d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.744792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1054275-d8d7-4729-9a86-c455b69d3c26 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.767194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.768533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.769134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.769427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.781821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.783105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.783595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.784083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:11.784184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.784951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:11.785274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.786723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffdb73-37da-4112-88b8-30b449dd4154 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:11.793311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:11.794214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:11.795040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:11.796167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 407/803] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:11.796235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.800987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 397/804] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:11.802481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.802976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.803451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.811843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:11.820475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:11.825065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:11.826329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:11.900770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 398/805] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:11.901917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:11.919066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a4f40-c7f3-4876-a3dd-2fa27fd7bf2e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:11.954000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:11.954893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:11.955180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:11.955462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:11.966200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:11.967737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:11.984762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:11.986527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:11.987493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:11.987493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:11.987493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:11.987493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:11.989029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 408/806] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:11.989971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.010134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffcdad-ebce-422d-b4d0-4bd882b5046e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.044618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.045317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.045599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.045869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.046609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:12.046913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.048605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f900133-6db8-4559-8626-b8a411a91f90 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.049996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.055510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.057903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 399/807] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:12.059139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05ac67-d68b-4496-be31-9b8732bcdf07 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.059424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05ac67-d68b-4496-be31-9b8732bcdf07 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.059699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05ac67-d68b-4496-be31-9b8732bcdf07 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.061094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 400/808] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:12.061608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05ac67-d68b-4496-be31-9b8732bcdf07 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.094381 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cdd-b935-4200-8787-5906ca1fedd0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.129188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.130078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.130354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.130628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/04be59c2cfa049629282356a9e209900/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.132559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=04be59c2cfa049629282356a9e209900, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.134024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.146751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 409/809] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:11 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/04be59c2cfa049629282356a9e209900/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:15:12.147748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.182886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05ac67-d68b-4496-be31-9b8732bcdf07 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.223761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.224711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.225031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.225341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.226093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.227489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:12.245331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:12.246586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:12.252964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 401/810] 10.222.0.79 () {56 vars in 1129 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:12.254096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.268815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e838161-ce33-4387-aa0d-0ea08254fd05 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.306198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.316150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.316822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.317167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.317439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.318165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:12.318472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.319847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a998ac1-4806-487e-b23a-06316615beeb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.324182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.333130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 410/811] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:12.334423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.334711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.334982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.351828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:12.353492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:12.380671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3719e9-fcea-4895-a755-45412eaf813f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.418195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.419176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.419564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.419953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.432900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 411/812] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:12.432900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.433189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.433543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:12.453009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:12.454311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:12.455415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:12.455415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:12.455415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:12.455415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:12.456777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 402/813] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:12.457593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.545837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792524b1-73bd-485c-813d-2bb3e55fda62 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.565920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.566407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b863a0e-4dd9-4f43-b0d1-06436da339fe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.581819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.587049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.587741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.588137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.588417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.589648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:12.589964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.591830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.603301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 412/814] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:12.603301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.607122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.607935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.608224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.608506 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.616368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.617868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:12.632794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:12.633743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:12.634625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 403/815] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:12.635104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.727866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1748714-feb1-4a68-aafc-cee691093a5a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.748472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d4e82-00dd-473d-9e16-966471bbe31f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.765703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.766565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.766939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.767222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.767890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.769302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:12.781702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:12.782860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:12.783688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 413/816] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:12.784499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.791923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.792602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.792916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.793197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.793933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:12.794246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.795699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.812359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 404/817] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:12.812359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.839819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.849364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.899374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f1fbc-8010-438a-92c2-56cf1db69dc6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.927109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77c99-ccec-4ecf-97d2-48a9b9ff6a69 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.935882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.936933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.937228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.937829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.939324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:12.939637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.941033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.950548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 414/818] 10.222.0.79 () {56 vars in 1124 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:12.950548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.964537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.965373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.965931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.966404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.967653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:12.968200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.970273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.983539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 405/819] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:12.984361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.061152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048ec2b-ddaf-4dd7-8f44-8915f3b59fda None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.072068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:13.086131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:13.095345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7b85-a1fc-4e76-8f8f-edcfe5999c5b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.103159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.103885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.104175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.104814 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.105561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] RBAC: Adding query filter params (name=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:13.106218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.107885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.112807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 415/820] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:13.117898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.131019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.131814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.132102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.132376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.140748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.142973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:13.158221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:13.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 406/821] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:13.160157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.233561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d29bd-8a54-4284-a4af-fb54f5f9fc17 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.270136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6235cf8-8f68-4fa6-bc00-b887ee6160e9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.271647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.272504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.273013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.273289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.274050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.275456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.308909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 416/822] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:13.308909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.319208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.320274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.320639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.321023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/f9fd37a822f04be2ab327efb57f59b0c/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.345174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=f9fd37a822f04be2ab327efb57f59b0c, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.347599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.349570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:13.358219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:13.367453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 407/823] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:13 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/f9fd37a822f04be2ab327efb57f59b0c/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:15:13.368665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.490730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd39fc0-debc-4977-aac6-cff477440a7d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.504770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd151-f50a-4ac6-9687-796185a65392 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.549868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.550566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.550855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.553526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.553526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:13.553526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.553927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.569417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.571480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.572384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.573190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.574808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] RBAC: Adding query filter params (type=network) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:13.575670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.578352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 408/824] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:15:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:13.579221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.589567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.597023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 417/825] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:13.603204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:13.778566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:13.792156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:13.798580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7d4ab-6d98-4605-9948-342190d249ef None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.835011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e74b7-d870-4999-a97d-a2eec4d745ca None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:13.871875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.872454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.873234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.873525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.898214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:13.898887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:13.899169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:13.899504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:13.900559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.901989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.959590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:13.962444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:13.967038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 418/826] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:14.069338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:14.108430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 409/827] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:15:14.118046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.244413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750e0de-fbe7-4635-bacd-4932b5e56458 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:14.283356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:14.284170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.284456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.284733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.295983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:14.297868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:14.305528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-93015f68-5ca9-4508-a2de-f8188a32ffa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:14.317527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:14.318536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:14.319437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:14.319437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:14.319437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:14.319437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:14.321918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 419/828] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:14.327426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.451128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e24f47-e1f6-4ad1-a0b1-66685fd423db None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:14.454850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae5edb-ed8e-4bd2-a6e7-0d271f2574fb None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.455454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae5edb-ed8e-4bd2-a6e7-0d271f2574fb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.455964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae5edb-ed8e-4bd2-a6e7-0d271f2574fb None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.458492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 420/829] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:14.464443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.465127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.465622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.487936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:14.489679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:14.498228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:14.498841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.499129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.499406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.500136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:14.500446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:14.501898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:14.514541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 410/830] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:14.520754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.541302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:14.574197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 421/831] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:14.606287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.637293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe6f4f-5b40-4584-9e6e-398a6fb01400 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:14.678202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:14.679175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.679564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.680042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.700956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:14.702766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:14.719345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdf527-14ca-451e-9687-acd7e481a6ce None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:14.720231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:14.721233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:14.722114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:14.724655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 411/832] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:14.732790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.749372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:14.757610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:14.758420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.758708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.758990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.778780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:14.789060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:14.790015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:14.791463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 422/833] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:14.796499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.830593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:14.856725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53136-7588-496c-8cd2-ba351f24e7ba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:14.894762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:14.895427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.895753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.896034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.896828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:14.897639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:14.898609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:14.913352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 412/834] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:14.919106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:14.927110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f625bd-d92d-41a8-89ec-dcae51c16a91 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:14.965657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:14.966738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.967230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.967709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.968698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:14.969356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:14.972153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:14.986384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 423/835] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:14.992060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:15.055787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7e108-907c-447b-a83b-4b400321e885 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.057721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.106018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.107640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.108398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.108912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.110765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=9918e21a598f4668b5fde7b173b647ac) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:15.111587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:15.113976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:15.118416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.122602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9631-e86b-40de-8559-84601a161f3b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.124325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 413/836] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/role_assignments?user.id=9918e21a598f4668b5fde7b173b647ac&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:15.165968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.167173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.167688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.168173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.182255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:15.184386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:15.207912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:15.208920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:15.209844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:15.209844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:15.209844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:15.209844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:15.212313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 424/837] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:15.217811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:15.340709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ab6d0-b13a-4789-bff4-aee561bbfcb3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.377633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.378310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.378600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.378885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.379621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:15.379934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:15.381343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:15.395180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 414/838] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:15.400406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:15.522659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459f1c-b19b-4509-9550-2c38068b508a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.524661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.560999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.562193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.562686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.563871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.572367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:15.574142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:15.589340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:15.590819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:15:15.591774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:15.594489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 425/839] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:15.602910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:15.669470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.726205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a1497-5bef-4862-a99a-4f796ae202de None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.762854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.763828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.764265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.764693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.765757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:15.766359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:15.769071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4945a4-dd28-40fe-88d9-c286157e267d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:15.787801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 415/840] 10.222.0.79 () {56 vars in 1135 bytes} [Wed Jul 20 19:15:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:15.793396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:15.804592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89741367-bcc3-4718-a6ab-687de7a0bbce None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.805100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89741367-bcc3-4718-a6ab-687de7a0bbce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.805603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89741367-bcc3-4718-a6ab-687de7a0bbce None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.807350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 416/841] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:15.813656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe5ede-5f05-4f07-90ba-518fc86b8dbd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.814021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe5ede-5f05-4f07-90ba-518fc86b8dbd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.814310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe5ede-5f05-4f07-90ba-518fc86b8dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.833483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cfbe5ede-5f05-4f07-90ba-518fc86b8dbd None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:15.834686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cfbe5ede-5f05-4f07-90ba-518fc86b8dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:15.916310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 417/842] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:15.917661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae9dc-b17f-4be6-8979-fd9e54a7a9c6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.917954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae9dc-b17f-4be6-8979-fd9e54a7a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.918228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae9dc-b17f-4be6-8979-fd9e54a7a9c6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.919632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 418/843] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:15.926165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.926696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.927184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.928672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7675d0-d1b8-463a-8fd6-cdfcfa3c863f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.945810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:15.947017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:15.966803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.967454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.967737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.968012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.995380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.002281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.023335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 419/844] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:16.024483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.053444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.055243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.146630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5b738-1a0c-424e-8cc1-1015671563f1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.196850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 426/845] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:16.198038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.207190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.208438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.208916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.209366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.226005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:16.240523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:16.241785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:16.242671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 420/846] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:16.243416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.289531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.296951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.329916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec7a9a-4c2b-4b26-a1ae-2241cdcb14d9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.365068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6698f5-8696-4f07-9f00-33ff973e524a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.370739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.371550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.371836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.372113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.373250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.375025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:16.388104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:16.389427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:16.390293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 427/847] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:16.391101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.406586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.408012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.408291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.408565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.420529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.422112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:16.440631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:16.441907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:16.447712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:16.447712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:16.447712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:16.447712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:16.447712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 421/848] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:16.447712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.513651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f7090-8950-4d74-a331-10a4952a279a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.541696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.542411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.552264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.552995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.553281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.553557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.553850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829abb-4a9c-4993-a5f7-9f0b8f8660b1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.554309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:16.554617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.555988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.565616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 428/849] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:16.566971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.591987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.592649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.592958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.593243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.593944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:16.594305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.596174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.601030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 422/850] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:16.601959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.694827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732aa30-040e-4457-9340-5b3f83fee97c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.716218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46db30-678e-4970-b646-0d3d9714c3e4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.731406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.732074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.732359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.732632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.733387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:16.733722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.735840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.748912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 429/851] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:16.749942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.757834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.758915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.759383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.759840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.771822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.773681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:16.792830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:16.793843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:16.794946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:16.794946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:16.794946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:16.794946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:16.801209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 423/852] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:16.802142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.811814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.813551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:16.867695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cbe79-61c3-42ed-b740-3e47bf006ed5 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.903175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.903839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.904123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.904434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.905136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:16.905443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.907483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:16.911690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dc98b-0316-48b1-bfda-3f32ceee1855 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:16.921144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 430/853] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:16.921144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:16.975237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:16.976087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:16.977304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:16.977594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:16.991780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:16.993679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:17.019483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:17.020460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:17.021574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 424/854] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:17.026982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.062543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262cb3-7e4a-4e57-bb54-da50fda20e50 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.082141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:17.108134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.108819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.109138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.109418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.110191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:17.110503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.111878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.112867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:17.124576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 431/855] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:17.134479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.152934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945ab4-a1f4-4826-9353-d82282e374e4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.194806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.195566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.195924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.196278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.197172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:17.197555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.199066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.211914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 425/856] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:17.218348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.269688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5f4a-d1ac-416e-b0cb-4aa6acb316c0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.315727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.316959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.317435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.317812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.327059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.329465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.337446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70cc2-a6cd-4428-9233-dcf585200237 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0ded6912-12ad-4028-b0c8-8d811ac8a5b0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:17.346602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:17.347723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:17.348846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:17.352381 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 432/857] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:17.353812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4e557-64d3-4d9c-9d6c-f120da63ffaf None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.354110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4e557-64d3-4d9c-9d6c-f120da63ffaf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.354373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4e557-64d3-4d9c-9d6c-f120da63ffaf None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.355775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 433/858] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:17.356558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4e557-64d3-4d9c-9d6c-f120da63ffaf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.376282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:17.386132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.386779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.387075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.387354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.388192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=04be59c2cfa049629282356a9e209900) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:17.388514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.389973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246b8c2-5fdd-478a-89d1-e5b2f7f2c3e6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.390724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:17.397058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 426/859] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/role_assignments?user.id=04be59c2cfa049629282356a9e209900&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:17.399598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.400093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.400570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.428036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:17.430118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:17.486900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4e557-64d3-4d9c-9d6c-f120da63ffaf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.521374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 427/860] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:17.541263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.543591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.545395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.545686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.545972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.546894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:17.547212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.548787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.574796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 434/861] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:17.581070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.684269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbd50a-9765-4621-a22d-9264657307ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.709320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:17.726739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.727575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.727861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.728135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.728806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.730232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:17.744246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:17.745356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:17.746176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 428/862] 10.222.0.79 () {56 vars in 1132 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:17.751524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.844987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:17.859429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2807-c9bf-47fd-8f0b-92faca043227 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.872590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1dae5-e8cc-45df-9159-209164235602 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:17.907441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.908648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.909184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.909680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.910631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:17.911303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:17.911591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:17.911871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:17.912586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:17.912915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.914310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.930454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 429/863] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:17.933200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:17.994057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:17.995525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:17.996703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:18.053080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8df3-4378-46db-b51a-ffedf5d444a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.090388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:18.099396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.100928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.101556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.102114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.103060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:18.103556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:18.109039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 435/864] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:15:18.109573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:18.110797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:18.119698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 430/865] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:18.237452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1194d53-5da6-4adc-b796-63d9e81ff931 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.278160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.278919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.279206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.279482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.291064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:18.292535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:18.312096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:18.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:18.314525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:18.314525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:18.314525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:18.314525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:18.314525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 436/866] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:18.320993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:18.428165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400919-7bc5-4b5f-979e-c91a1e762fe8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.441695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:18.463358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.464026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.464311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.464589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.465347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:18.465656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:18.467113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:18.481440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 431/867] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:18.487673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:18.534377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:18.593442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5ff53-6fd5-4ad2-9af5-7e1ebbf519f9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.631163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.631972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.632256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.632534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.640004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:18.642101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:18.657674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:18.658665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:18.659990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 437/868] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:18.664906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:18.771185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f20a63-371d-4fd3-ba4b-bf26fedf4f5b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.808282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.809055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.809396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.809680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.810468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:18.810780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:18.812196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bc3dc-936d-4ee9-a16f-e63210ac74ea None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:18.825414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 432/869] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:18.831553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:18.936561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb2f8b-7a4e-44b5-b2e2-1d98d32b3453 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.938773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:18.973500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.974085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.974368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.974643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.975492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=f9fd37a822f04be2ab327efb57f59b0c) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:18.975803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:18.977284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add1b74-5136-4dfa-83d7-9a22910f4c4b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:18.984171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 438/870] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:18 2022] GET /identity/v3/role_assignments?user.id=f9fd37a822f04be2ab327efb57f59b0c&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:19.136540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28dcfd-8b04-404f-bc5a-ce6fbd6e64ad None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:19.136938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28dcfd-8b04-404f-bc5a-ce6fbd6e64ad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:19.137150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28dcfd-8b04-404f-bc5a-ce6fbd6e64ad None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:19.138667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 433/871] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:19.145095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2086fd-a434-4a3c-857c-f8a47147b91b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:19.145388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2086fd-a434-4a3c-857c-f8a47147b91b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:19.145666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2086fd-a434-4a3c-857c-f8a47147b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:19.165216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ac2086fd-a434-4a3c-857c-f8a47147b91b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:19.166747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ac2086fd-a434-4a3c-857c-f8a47147b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:19.241187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 439/872] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:19.265035 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28dcfd-8b04-404f-bc5a-ce6fbd6e64ad None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:19.302122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:19.381132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28dcfd-8b04-404f-bc5a-ce6fbd6e64ad None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:19.415982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad58e24-6f87-461e-9195-49abf01114b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:19.416658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58e24-6f87-461e-9195-49abf01114b0 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:19.416968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58e24-6f87-461e-9195-49abf01114b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:19.417246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58e24-6f87-461e-9195-49abf01114b0 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:19.417971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad58e24-6f87-461e-9195-49abf01114b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:19.419335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad58e24-6f87-461e-9195-49abf01114b0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:19.437050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 434/873] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:19 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:19.591931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a01f7-60f6-4022-8480-cea2aa4b0939 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:19.592260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a01f7-60f6-4022-8480-cea2aa4b0939 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:19.592483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a01f7-60f6-4022-8480-cea2aa4b0939 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:19.594297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 440/874] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:19.600256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:19.600556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:19.600835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:19.617953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:19.619147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:19.701820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 435/875] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:19.725254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a01f7-60f6-4022-8480-cea2aa4b0939 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:19.829098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:19.838142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a01f7-60f6-4022-8480-cea2aa4b0939 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:19.875360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:19.876508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:19.877047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:19.877522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:19.878371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:19.879953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:19.892675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:19.893738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:19.894598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:19.897604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 441/876] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:19.904915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:19.926762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:20.015178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7eb2f-c8ea-41f2-9346-c3475b2d0d6e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:20.052940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:20.053974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:20.054478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:20.054968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:20.055934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:20.056442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:20.058039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3eb72-0eb3-43ce-993c-ecca3cf215d5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:20.063609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 436/877] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:20.069142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:20.176267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074095de-a1dc-4fc2-a4eb-a84db95ba071 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:20.211455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:20.212515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:20.213057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:20.213561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:20.214475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:20.214982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:20.216552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d2df3-b379-4dfb-a8ad-45ebe88e619e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:20.222482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 442/878] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:20.344113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899cd7a-2e5e-4e2e-8ac0-2d4e34c1d6d1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:20.345356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899cd7a-2e5e-4e2e-8ac0-2d4e34c1d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:20.345839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899cd7a-2e5e-4e2e-8ac0-2d4e34c1d6d1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:20.347841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 437/879] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:20.353263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd789535-3343-4cbe-8b42-f1d12fb315b4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:20.353770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd789535-3343-4cbe-8b42-f1d12fb315b4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:20.354189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd789535-3343-4cbe-8b42-f1d12fb315b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:20.373299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bd789535-3343-4cbe-8b42-f1d12fb315b4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:20.374492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bd789535-3343-4cbe-8b42-f1d12fb315b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:20.397747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:20.444936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 443/880] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:20.468574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899cd7a-2e5e-4e2e-8ac0-2d4e34c1d6d1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:20.556530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:20.577039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899cd7a-2e5e-4e2e-8ac0-2d4e34c1d6d1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:20.613653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c779ad4-1f2b-4bf0-b24f-6048afd87846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:20.614316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779ad4-1f2b-4bf0-b24f-6048afd87846 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:20.614598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779ad4-1f2b-4bf0-b24f-6048afd87846 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:20.614869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779ad4-1f2b-4bf0-b24f-6048afd87846 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:20.615592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c779ad4-1f2b-4bf0-b24f-6048afd87846 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:20.616968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c779ad4-1f2b-4bf0-b24f-6048afd87846 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:20.637467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 438/881] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:20 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:21.257878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583f242-c940-40ca-8368-b24d27ba0f04 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.258916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583f242-c940-40ca-8368-b24d27ba0f04 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.259440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583f242-c940-40ca-8368-b24d27ba0f04 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.261338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 444/882] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:21.267103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.267988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.268469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.286382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:21.287889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.371726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 439/883] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:21.396708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583f242-c940-40ca-8368-b24d27ba0f04 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.501869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583f242-c940-40ca-8368-b24d27ba0f04 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:21.539115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a335f-0a78-4da4-bb18-6a96a1581ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:21.540261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a335f-0a78-4da4-bb18-6a96a1581ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.540766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a335f-0a78-4da4-bb18-6a96a1581ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.541284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a335f-0a78-4da4-bb18-6a96a1581ca5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.552494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a335f-0a78-4da4-bb18-6a96a1581ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ea3a335f-0a78-4da4-bb18-6a96a1581ca5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:21.561122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:21.562043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:21.563438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:21.566136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 445/884] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:21.572231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.601822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed902cf1-6945-430b-abb9-02ee356f81ae None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.602139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed902cf1-6945-430b-abb9-02ee356f81ae None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.602383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed902cf1-6945-430b-abb9-02ee356f81ae None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.607526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 446/885] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:21.609170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.609468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.611116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.627342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:21.628547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.645953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.651457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.705766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7969af7-45c6-4297-8835-22f430f3b518 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:21.716693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 447/886] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:21.740396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.747968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:21.749006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.749500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.749983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.751453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:21.752018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:21.753707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:21.763136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 440/887] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:21.770466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.874143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633d09-cf23-489b-92f1-0c9d03938b07 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:21.882172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81278fd0-6aeb-4911-bf2f-d844c10884e4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:21.913796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:21.914595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.914874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.915245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.915901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:21.917299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:21.924673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.928445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:21.929459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.929912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.930197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:21.930937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:21.931966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:21.932986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 448/888] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:21.941860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.942709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:21.944165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:21.955624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:21.968643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:21.970158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:21.971376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:21.971376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:21.971376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:21.971376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:21.972962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 441/889] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:21.976650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.073878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d42d88-86fa-4f56-bf02-e1e47257ae37 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.113000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b8bfc-b2bb-4573-9175-904e8bd16990 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.122043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.122990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.123498 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.123996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.125987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:22.127013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.129113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.136509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 449/890] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:22.144309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.158730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.159492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.159900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.160274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.161112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:22.161505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.162971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.182365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 442/891] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:22.183707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.230165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.240617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.276542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839679e-ca29-4926-b6fc-f623a488382c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.292142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdd65b-cf6e-4c24-9d6d-9f35b9c38d38 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.316704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.317737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.318304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.318687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.319413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:22.319721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.321150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd48423-92e3-4dec-9e09-76a9aee21b48 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.326864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 450/892] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:22.328043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a004d-91cd-4547-860b-d68d50367ac5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.328993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a004d-91cd-4547-860b-d68d50367ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.329167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a004d-91cd-4547-860b-d68d50367ac5 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.330751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 451/893] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:22.331408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.332850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.333293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.333537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a004d-91cd-4547-860b-d68d50367ac5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.333838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.342212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.343648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ebd5b500-d38d-480f-95e1-778b481ae860 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:22.359639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:22.361485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:22.362445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 443/894] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:15:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:22.363597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.363597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.363769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.381330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:22.382510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:22.449222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a004d-91cd-4547-860b-d68d50367ac5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.453134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 444/895] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:22.458066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.485764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.486400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.486682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.486946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.487743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] RBAC: Adding query filter params (interface=public, service_id=05a8621043f1489aa6d2c6d05bfa019e, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:22.488053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.489676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.494533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.504303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 452/896] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:22 2022] GET /identity/v3/endpoints?service_id=05a8621043f1489aa6d2c6d05bfa019e&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:22.505173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.508814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.582559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5660-d25c-469c-9dcd-43b27ce3417e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.616009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc97d6-23cc-46ae-882b-86b654d1fe74 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.618390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.619082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.619371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.619647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.620494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:22.620809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.622273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.642102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 445/897] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Jul 20 19:15:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:22.642939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.659538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.660198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.660488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.660770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.661545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.662926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.676921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 453/898] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:22 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:22.677581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:22.766788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.780466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:22.783713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e7879-6fd1-43a9-9f13-cdc33f0ca654 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.842779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5020bc-3b09-44f6-9a56-1367516d0deb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:22.849729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3542e-3855-4445-ba0f-c058ea4009bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.850406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3542e-3855-4445-ba0f-c058ea4009bd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.850687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3542e-3855-4445-ba0f-c058ea4009bd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.850962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3542e-3855-4445-ba0f-c058ea4009bd None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.851638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3542e-3855-4445-ba0f-c058ea4009bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.853032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3542e-3855-4445-ba0f-c058ea4009bd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.859360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 446/899] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:22 2022] GET /identity/v3/services => generated 1497 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:15:22.861624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeb072-a1b8-44f6-b5dc-685bd93bd56e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.861914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeb072-a1b8-44f6-b5dc-685bd93bd56e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.862206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeb072-a1b8-44f6-b5dc-685bd93bd56e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.863486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 447/900] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:22.871087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.871684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.871974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.891132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:22.892418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:22.893916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:22.894832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:22.895121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:22.895398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:22.986245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:22.988322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:22.996910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 448/901] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:23.015093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.060627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.116482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.129924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 454/902] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:23.135086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.143504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f25e5a-b357-4011-8aa1-9550944228f7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.187115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.188530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.188999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.189496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.190174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.191556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:23.204700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:23.205801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:23.206722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 449/903] 10.222.0.79 () {56 vars in 1132 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:23.214040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.252406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd5a4-d70b-4864-81ee-654622ea511e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.292516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.294061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.294509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.294997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.306923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.308743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:23.329544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:23.330497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:23.331412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:23.331412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:23.331412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:23.331412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:23.333864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 455/904] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:23.334619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d7df-aab4-4a72-96ee-789ad3a7e6db None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.341352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.353778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.385848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.386519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.386806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.387084 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.387800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:23.388108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.389510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.397426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 450/905] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:23.402930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.418652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.471017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5ba9-24bc-4eea-a334-876253e9fb67 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.512610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.514140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.514645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.515128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.516760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:23.517344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.519017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.528967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef3bb-c65d-4d45-93c0-898e2b8b48d3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.535297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 456/906] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:23.544299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.567002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.567825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.568208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.568586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.569500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:23.569908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.571402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.576933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 451/907] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:23.584334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.620908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.660791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb251af8-3ecd-4837-a9f7-f082243f8fac None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.690186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.709582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.710712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.711200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.712248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.723174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.725189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.725699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0caaf3a-3090-4eff-9563-c5ff0383dc91 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:23.745569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:23.746520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:23.747392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:23.750275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 457/908] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:23.756169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.768810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.769282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.769642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.769999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.770891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] RBAC: Adding query filter params (interface=public, service_id=b2cc871580894b1da522735c257e1951, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:23.771281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.772730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.783442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 452/909] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/endpoints?service_id=b2cc871580894b1da522735c257e1951&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:23.785449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.876248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e580be0-df99-429d-9613-d9bd32b9b886 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.901348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7e0a4-cfa4-4a65-8a99-fc50b6f874e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:23.908626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.919072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.919841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.920262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.920641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.921641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:23.922037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.924235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.942270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 458/910] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:23.943334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:23.945782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fce7ac-95bd-4495-a84f-8b87a7d04241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:23.946787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce7ac-95bd-4495-a84f-8b87a7d04241 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:23.947359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce7ac-95bd-4495-a84f-8b87a7d04241 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:23.947848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce7ac-95bd-4495-a84f-8b87a7d04241 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:23.948755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fce7ac-95bd-4495-a84f-8b87a7d04241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:23.950355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fce7ac-95bd-4495-a84f-8b87a7d04241 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:23.952545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:23.965039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 453/911] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:15:24.069981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0ba92-5f9b-4b1d-942a-b8ec24133200 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:24.110519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:24.111247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.111539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.111819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.112666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] RBAC: Adding query filter params (role.id=2aab0e07f8ec44a7a5d961b8e00c62ca, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=f9fd37a822f04be2ab327efb57f59b0c) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:24.113083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:24.114543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64119bbb-47e5-442f-92eb-c217fa5f2e36 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:24.121692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 459/912] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:23 2022] GET /identity/v3/role_assignments?user.id=f9fd37a822f04be2ab327efb57f59b0c&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=2aab0e07f8ec44a7a5d961b8e00c62ca => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:24.871917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4247410-0b32-4415-bf7f-3eb2679cb520 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.872462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4247410-0b32-4415-bf7f-3eb2679cb520 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.872462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4247410-0b32-4415-bf7f-3eb2679cb520 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.874132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 454/913] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:24.879942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8fb29-5f13-41ba-8ec5-efea0f2e8681 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.880399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8fb29-5f13-41ba-8ec5-efea0f2e8681 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.880575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8fb29-5f13-41ba-8ec5-efea0f2e8681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.898209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-91a8fb29-5f13-41ba-8ec5-efea0f2e8681 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:24.899411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-91a8fb29-5f13-41ba-8ec5-efea0f2e8681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:24.952573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:24.974379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 460/914] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:24.996895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4247410-0b32-4415-bf7f-3eb2679cb520 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.052621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9f0b5-ffbb-46e2-af1e-849e05d52f52 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.052925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9f0b5-ffbb-46e2-af1e-849e05d52f52 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.053209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9f0b5-ffbb-46e2-af1e-849e05d52f52 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.054862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 461/915] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:25.059990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.060281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.060557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.079342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:25.080503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:25.118162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4247410-0b32-4415-bf7f-3eb2679cb520 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.160310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.161352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.161538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.161773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.162488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.163910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:25.177447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:25.178493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:25.179332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:25.181672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 462/917] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:25.184753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 455/917] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:25.185048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.207613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.279049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:25.316634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31d1c6-8c13-4847-8add-468c1b364979 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.332355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:25.333360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e328508-6196-4b42-8d2b-de60930fd698 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.364360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.365656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.366403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.366884 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.368525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:25.369065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.370985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.373594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.374539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.374897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.375311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.376693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.378314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.380453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 463/918] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:25.388941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:25.401432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:25.402769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:15:25.403727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:25.407678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 456/919] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:25.413705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.512377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adc817-9901-4568-b75b-780b5125e172 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.550089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d046c8-eb2c-4c61-a6fe-e3e50405263f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.553166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.554309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.554825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.555294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.556255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:25.556758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.558360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.564445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 464/920] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:25.570497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.585622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:25.596687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.597392 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.597673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.597943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.598744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:25.599086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.600553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.611157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 457/921] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:25.612769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.620702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:25.684558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83bcef-2119-418a-b83e-7174b57033dd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.725231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.726105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.726528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.726865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.727818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] RBAC: Adding query filter params (interface=public, service_id=5d0023d7f66e4b4086fe27234f948a27, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:25.728254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.729248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1e32-3123-424e-bf84-7a89587e0206 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.729878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.740708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 465/922] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/endpoints?service_id=5d0023d7f66e4b4086fe27234f948a27&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:25.741839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.776262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.777059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.777350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.777881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.778941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:25.779261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.780948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.787959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 458/923] 10.222.0.79 () {56 vars in 1114 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:25.793735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:25.866776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:25.885713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298342-1ca9-46bc-8384-cdde5d712f37 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.890266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:25.927979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064f23d-254e-4a54-968f-dfb8cae6a08f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:25.943016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc08a-f82a-48bd-8b72-93f734138d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.944277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc08a-f82a-48bd-8b72-93f734138d18 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.944747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc08a-f82a-48bd-8b72-93f734138d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.945243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc08a-f82a-48bd-8b72-93f734138d18 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.946265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc08a-f82a-48bd-8b72-93f734138d18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.948501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc08a-f82a-48bd-8b72-93f734138d18 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:25.956860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 466/924] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:25 2022] GET /identity/v3/services => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:15:25.975347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7afaa-504b-4925-9585-be332fc85d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:25.976406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7afaa-504b-4925-9585-be332fc85d40 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:25.976927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7afaa-504b-4925-9585-be332fc85d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:25.977401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7afaa-504b-4925-9585-be332fc85d40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:25.978411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7afaa-504b-4925-9585-be332fc85d40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:25.980058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7afaa-504b-4925-9585-be332fc85d40 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:26.006119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 459/925] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:15:26.138683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b6820-3239-4d83-8313-377b55578f52 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.138991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b6820-3239-4d83-8313-377b55578f52 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.139235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b6820-3239-4d83-8313-377b55578f52 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.140662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 467/926] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:26.147754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516ad4-6bf3-4e43-9953-1336c4bb9204 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.148045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516ad4-6bf3-4e43-9953-1336c4bb9204 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.148323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516ad4-6bf3-4e43-9953-1336c4bb9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.167266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f8516ad4-6bf3-4e43-9953-1336c4bb9204 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:26.168474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f8516ad4-6bf3-4e43-9953-1336c4bb9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:26.248533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 460/927] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:26.253359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afb117-75d7-41e0-b098-f6787af98764 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.253645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afb117-75d7-41e0-b098-f6787af98764 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.253925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afb117-75d7-41e0-b098-f6787af98764 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.255545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 468/928] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:26.260631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.260952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.261241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.275477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afb117-75d7-41e0-b098-f6787af98764 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.279692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:26.280947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:26.313589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:26.365292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 461/929] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:26.388616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.405583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afb117-75d7-41e0-b098-f6787af98764 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.442905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:26.443731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.444016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.444293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.445156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:26.446409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:26.460048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:26.461107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:26.462309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 469/930] 10.222.0.79 () {56 vars in 1132 bytes} [Wed Jul 20 19:15:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:26.466879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.501873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:26.535909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b210f6-3726-43c0-99f5-64c415afc831 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.584427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daa4c7-e5ed-4973-869e-36a387c5e08b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.586702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:26.588618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.589105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.589530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.603752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:26.615914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:26.617112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:26.618164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:26.620978 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 462/931] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Jul 20 19:15:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:26.627127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.628833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:26.629713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.630252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.630676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.631682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:26.631997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:26.633424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:26.643901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 470/932] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:26.645107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.670919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:26.757706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:26.763948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98923c55-d788-4440-9dd0-60b6ca529281 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.769479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4431c-acfc-499d-abdc-9cd1e4dc19ab None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.803722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:26.804489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.804858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.805224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.806111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:26.806503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:26.808159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:26.810611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:26.811458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.811907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.812336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:26.813750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:26.814160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 471/933] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:26.814505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:26.815882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:26.820989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.824558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 463/934] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:15:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:26.830818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:26.927940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:26.944955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cde8-e23c-4d6b-825f-f9a4da48cb6f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.970949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f74e6c-18ec-4653-bf58-e022c33540e7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:26.989508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:26.990307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:26.990590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:26.990865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.005886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.006903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.007453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.007562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.007833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.008109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.008860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.010495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:27.030335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:27.031289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:27.033310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:27.033310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:27.033310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:27.033310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:27.037050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 464/935] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:27.042097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:27.043131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 472/936] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:27.165067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17ef4c-8d65-4302-ae51-7aeb761dbf10 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:27.209608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.210181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.210521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.210963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.211813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:27.212132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.213625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.222537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.230031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 465/937] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:27.238044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:27.264677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.355701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdaaba-0df1-41bd-9f7b-79b1d753d69c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:27.391135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.392167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.392454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.392791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.401219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.402956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:27.420103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:27.421137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:27.422766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 473/938] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:15:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:27.430636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:27.550720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc67046-d6f3-46c6-9363-4ddd012501f5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:27.592583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.593664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.594268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.594748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.595865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:27.596391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.598109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.619320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 466/939] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Jul 20 19:15:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:27.626072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:27.728435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.747829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ed23d-b52e-496e-a76a-e3aa5fea3274 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:27.788030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.788911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.789380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.789659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.802865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.804249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:27.823311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:27.827966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:27.831354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:27.831354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:27.831354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:27.831354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:27.831354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 474/940] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:27.832002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:27.894549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:27.951060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a301fc-d20e-4917-bbfc-66c786d08d77 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:27.988387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:27.989100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:27.989462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:27.989661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:27.990404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:27.990713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:27.992148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.009087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 467/941] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:28.011669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.130332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4be8a-245e-4487-b677-1235a812acc8 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:28.160464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:28.169243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:28.170361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.170952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.171232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.182198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:28.183726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3d81a481-417f-42da-8f3e-da9d6ec35cc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:28.205020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:28.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:28.207557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:28.208541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 475/942] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:28.214011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.314866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a0c8-a14a-4ead-9750-7e5597d8ded5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.315160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a0c8-a14a-4ead-9750-7e5597d8ded5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.315571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906a0c8-a14a-4ead-9750-7e5597d8ded5 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.317466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 476/943] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:28.323716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.324072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.324431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.325419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b17d6-8528-497e-9962-5f75817b0ed1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:28.350532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:28.354853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:28.357333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:28.385369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:28.386654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.387139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.387482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.388376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:28.388752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:28.390430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.405394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 468/944] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:28.411412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.456797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 477/945] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:28.479691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.532489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8080d7-f671-49a8-a6ce-b513b10b7140 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:28.571343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b5951-9331-4ff1-a505-bee88d5b6135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:28.572356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b5951-9331-4ff1-a505-bee88d5b6135 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.572640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b5951-9331-4ff1-a505-bee88d5b6135 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.572939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b5951-9331-4ff1-a505-bee88d5b6135 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/f9fd37a822f04be2ab327efb57f59b0c/roles/2aab0e07f8ec44a7a5d961b8e00c62ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.575413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b5951-9331-4ff1-a505-bee88d5b6135 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=f9fd37a822f04be2ab327efb57f59b0c, role_id=2aab0e07f8ec44a7a5d961b8e00c62ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:28.576858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b5951-9331-4ff1-a505-bee88d5b6135 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.593173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd91b53-bdaf-4fba-aa08-32532be28a34 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:28.608921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 469/946] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:28 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/f9fd37a822f04be2ab327efb57f59b0c/roles/2aab0e07f8ec44a7a5d961b8e00c62ca => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:15:28.608921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52f00-8bf8-42be-ba0b-98cd4d94145f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.608921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52f00-8bf8-42be-ba0b-98cd4d94145f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.608921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c52f00-8bf8-42be-ba0b-98cd4d94145f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.609358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 470/947] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:28.612497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.613071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.613598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.623140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:28.643033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:28.645049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:28.646196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:28.647336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.647923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.648418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.649450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:28.651238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:28.667179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:28.668256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:28.669163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:28.677510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 478/948] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:28.678949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.682190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:28.739257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 471/949] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:28.774040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.835962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b7707-fd73-4a34-a3a1-47cbbc71305c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:28.876487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:28.877215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.877531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.877810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.878519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:28.878837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:28.880354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.890342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 479/950] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:28.896844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:28.927006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf9acc-cf69-426f-a77f-c65129035f61 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:28.947024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:28.975732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:28.976469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:28.976674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:28.977746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:28.978501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:28.979941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:28.981365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING keystone.server.flask.common [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 19:15:29.003142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:15:29.003142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 19:15:29.003142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [parameters: {'id': 'c2ff1a8f10cc4ed5b75a1164914f7862', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 19:15:29.003142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:15:29.012100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d45e2-bae8-4e9d-b8f3-055639bfb049 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2ff1a8f10cc4ed5b75a1164914f7862.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2ff1a8f10cc4ed5b75a1164914f7862. Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:15:29.012811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 19:15:29.013661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() Jul 20 19:15:29.014630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:15:29.015556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:15:29.023479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:15:29.025612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': 'c2ff1a8f10cc4ed5b75a1164914f7862', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.028018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:29.029892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c2ff1a8f10cc4ed5b75a1164914f7862. Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 472/951] 10.222.0.79 () {60 vars in 1140 bytes} [Wed Jul 20 19:15:28 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:29.032682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.055807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.073554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.074290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.074579 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.074854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.075573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:29.075882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.077288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.085015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 480/952] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:29.089843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.155211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683093f-b2e1-40a1-a6b3-3c14e2ff31d4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.191545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.192458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.192750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.193074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.200523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.202260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.216651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89498f6-8a52-4e67-989c-c3166d0a3c66 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:29.218470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:29.220575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:29.221564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 473/953] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:29.228281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.229257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.261715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463dcbe-8a12-4694-8ff1-4d6e98c7683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.262672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463dcbe-8a12-4694-8ff1-4d6e98c7683b None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.262974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463dcbe-8a12-4694-8ff1-4d6e98c7683b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.263256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463dcbe-8a12-4694-8ff1-4d6e98c7683b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.264275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463dcbe-8a12-4694-8ff1-4d6e98c7683b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.266205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463dcbe-8a12-4694-8ff1-4d6e98c7683b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.297511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 481/954] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:15:29.340224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.364529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88c7d8-cf02-498b-95da-94a858f8f18a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.417675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.418713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.419204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.419749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.420785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:29.421263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.423868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ec41-7c6f-4388-ab60-385d93e05de1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.446373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 474/955] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:29.469575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06922b7f-bcdb-49e2-8c93-b61e7787041a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.469874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06922b7f-bcdb-49e2-8c93-b61e7787041a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.470146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06922b7f-bcdb-49e2-8c93-b61e7787041a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.473167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 482/956] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:29.478236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.478529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.478811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.497867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:29.499459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:29.585068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 475/957] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:29.604277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06922b7f-bcdb-49e2-8c93-b61e7787041a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.719904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06922b7f-bcdb-49e2-8c93-b61e7787041a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.762476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.763665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.764154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.764652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.765550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.768241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:29.782397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:29.783537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:29.784616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:29.787509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 483/958] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:29.793342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.871569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.911282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3b3af-13cf-4542-922d-ebbd2366201f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.948513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.949552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.949983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.950403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.951280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:29.951733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.953869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f7f57-7061-4630-9319-3e49be6aa850 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.959783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 476/959] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:29.965972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.999436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:30.074337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75695b4e-f931-40c7-b4c8-8f0e6e2b7f39 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:30.115913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:30.117106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:30.118852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:30.118852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:30.118852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:30.119134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:30.120310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04b6ba-948d-4072-8645-2935f13ccb12 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:30.131340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 484/960] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:30.831329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da12b9-3eb9-4b4c-9547-99867338aa48 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:30.831844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da12b9-3eb9-4b4c-9547-99867338aa48 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:30.831951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da12b9-3eb9-4b4c-9547-99867338aa48 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:30.833544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 477/961] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:30.838968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:30.839262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:30.839545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:30.856418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:30.857634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:30.931480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 485/962] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:30.954900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da12b9-3eb9-4b4c-9547-99867338aa48 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.060185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da12b9-3eb9-4b4c-9547-99867338aa48 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:31.092720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:31.101545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:31.102549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.102935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.103322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.118977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:31.128281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:31.129273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:31.130111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:31.136934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 478/963] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Jul 20 19:15:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:31.139020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.169430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:31.250106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb273-c087-414e-bb17-c9b72d8fb644 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:31.286978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:31.287682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.287978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.288256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.289076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:31.289386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:31.291220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:31.301105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 486/964] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:31.306515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.416644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a6c6d-09fa-4f6f-8029-740fe4c3625f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:31.452310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:31.453166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.453479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.454085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.469569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:31.471032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-98cd8320-027c-4565-ae64-47e83b5c3d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:31.489567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:31.490501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:31.491400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:31.491400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:31.491400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:31.491400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:31.491400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 479/965] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:31.496638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.582178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0900-de62-4331-a810-1a86b96def37 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.582538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0900-de62-4331-a810-1a86b96def37 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.582739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0900-de62-4331-a810-1a86b96def37 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.584370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 480/966] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:31.589254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.589541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.589816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.591392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:31.604273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e56de0-490e-487b-9f6f-6fe36240fdff None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:31.608308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:31.610379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:31.647827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:31.648512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.648781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.649101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.649833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:31.650144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:31.652964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:31.657885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:31.672860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 487/967] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:31.678494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.701016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 481/968] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:31.716797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.789355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87b4a-9203-4be7-a244-b49db79429eb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:31.823434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3d853-1f32-4880-9c2f-051d4bbdecdd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:31.826858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:31.827927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.828510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.829043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.836846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:31.838561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3481b978-0c13-4375-81c3-45158e4e766d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:31.854361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:31.855325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:15:31.856260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:31.858595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:31.859383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.860164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 488/969] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:31.860286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.860473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.861232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:31.862201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc3339-89c6-4a7b-a003-52783bc87841 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.862621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:31.862977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc3339-89c6-4a7b-a003-52783bc87841 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.863412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc3339-89c6-4a7b-a003-52783bc87841 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.865291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 489/970] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:31.866017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc3339-89c6-4a7b-a003-52783bc87841 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.875117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-79b0bbba-20b3-4231-9bc6-67aaae040213 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:31.880654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:31.881873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:31.882778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 482/971] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:31.883614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:31.884224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:31.884224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:31.900741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:31.901989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:31.935781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:31.981007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 483/972] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:31.981854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:31.998266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc3339-89c6-4a7b-a003-52783bc87841 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.038804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.039469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.039750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.040028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.040855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:32.041183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.042608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.066201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 490/973] 10.222.0.79 () {56 vars in 1149 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:32.066201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.094565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058d238-8ed7-4aa8-82c1-3d0474deaa20 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.127870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.129861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.130535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.130821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.131097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.131848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:32.132122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.133804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.138422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 484/974] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:32.138422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.175715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe71d-d7ed-42f0-81f5-85fcf04eba87 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.178803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.213499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.214174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.214454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.214724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.215414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.216840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.218162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING keystone.server.flask.common [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 19:15:32.230427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.sql.core [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:15:32.230427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 19:15:32.230427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [parameters: {'user_id': 'd42a7b7935ae489080560139900060ea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 19:15:32.230427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80250) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application result.read() Jul 20 19:15:32.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:15:32.248474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application yield resource Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise self.value Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application session.commit() Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:15:32.249854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:15:32.252153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:15:32.253878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application result.read() Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:15:32.257483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd42a7b7935ae489080560139900060ea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:32.259528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 19:15:32.262117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 491/975] 10.222.0.79 () {60 vars in 1131 bytes} [Wed Jul 20 19:15:32 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:32.265368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.267747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc54c1-a53d-4338-b853-a8cb39703795 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.296605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e9607-a10e-4b30-aa14-9d36712e4493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.297387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e9607-a10e-4b30-aa14-9d36712e4493 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.297670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e9607-a10e-4b30-aa14-9d36712e4493 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.297945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e9607-a10e-4b30-aa14-9d36712e4493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.355229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b9924-916e-409b-bdd8-44d069bd9d98 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.365475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e9607-a10e-4b30-aa14-9d36712e4493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.366932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e9607-a10e-4b30-aa14-9d36712e4493 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.381089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.391399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.392071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.392355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.392636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.393383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:32.393694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.395256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.397337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.403564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 492/976] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:32.404427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.469790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 485/977] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:32.471121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8bf14-eb0f-4bdb-8e92-d24b6f0816dd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.471412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8bf14-eb0f-4bdb-8e92-d24b6f0816dd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.471689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8bf14-eb0f-4bdb-8e92-d24b6f0816dd None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.472984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 486/978] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:32.473669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8bf14-eb0f-4bdb-8e92-d24b6f0816dd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.513558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cffdcc-13d8-45f8-ac92-acaa8d85e8fd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.548668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.549495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.549782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.550067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.557720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.559161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:32.573719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:32.574819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.575781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 493/979] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:32.576489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.586242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8bf14-eb0f-4bdb-8e92-d24b6f0816dd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.616492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.622275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.623004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.623357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.623704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.624571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] RBAC: Adding query filter params (interface=admin, service_id=5d0023d7f66e4b4086fe27234f948a27, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:32.624899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.626297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1745d3f-96ff-446a-8723-a19da9a701f8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.627890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.631758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 487/980] 10.222.0.79 () {56 vars in 1249 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/endpoints?service_id=5d0023d7f66e4b4086fe27234f948a27&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:32.632966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.633251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.633522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.649201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:32.650382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:32.682395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b34a7-abe1-4423-b54b-c97b5d40a0eb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.720317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 488/981] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:32.721171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.721865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.722005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.722240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.722583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.736871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.738358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:32.757259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:32.758491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:32.759897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:32.759897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:32.759897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:32.759897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:32.759897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 494/982] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:32.759897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.829283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3a993-99e9-437c-a725-1c134fb651e0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.855764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.861783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c5132-4821-4ec4-9780-48d11fdb400a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.863318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.865135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.865837 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.866099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.866381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.867233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:32.867491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.868966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.896804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.897508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.897789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.898059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.898763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.900119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.904751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 489/983] 10.222.0.79 () {56 vars in 1101 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:32.905587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.909197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 495/984] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:15:32.909978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:33.057375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d0e10-ca64-4ac9-aabd-06218eb1ed35 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:33.070431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323cb7a-372f-4e99-ad4c-6cba57fe791c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:33.103138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:33.104113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.104371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.104647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.105586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:33.106224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:33.107601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:33.114507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:33.116305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.116779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.117341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.118546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:33.120238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:33.121305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 496/985] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:33.128044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:33.144496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 490/986] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:32 2022] POST /identity/v3/services => generated 242 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:33.169744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:33.267077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4188a-68a2-4d27-9af0-b5457486bb6c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:33.313943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:33.314744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.315024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.315295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.323227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:33.324671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:33.339847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:33.340767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:33.341708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 497/987] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:33.347128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:33.402358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:33.467112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855a253-7476-4c31-b9cb-6a792b5d96ed None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:33.505003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:33.505677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.505961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.506238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.507030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:33.507344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:33.508779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a65961-cc17-4bdb-aa97-426d1d4944f4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:33.526523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 491/988] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:33.528443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:33.616708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:33.636760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26c3e-9256-4a08-9ff8-dff7782b3a01 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:33.675740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:33.676494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:33.676772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:33.677087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:33.677924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] RBAC: Adding query filter params (role.id=2aab0e07f8ec44a7a5d961b8e00c62ca, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=f9fd37a822f04be2ab327efb57f59b0c) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:33.678235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:33.679715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185e5e4-fc87-4150-84c8-085973e75579 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:33.692992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 498/989] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:33 2022] GET /identity/v3/role_assignments?user.id=f9fd37a822f04be2ab327efb57f59b0c&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=2aab0e07f8ec44a7a5d961b8e00c62ca => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:35.037439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18fb2f-42d7-410d-8583-bde251b01189 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.037979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18fb2f-42d7-410d-8583-bde251b01189 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.037979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18fb2f-42d7-410d-8583-bde251b01189 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.039583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 492/990] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:35.045011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19141-5a9d-496f-b025-f514344b7129 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.045459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19141-5a9d-496f-b025-f514344b7129 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.045562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19141-5a9d-496f-b025-f514344b7129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.064441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6ba19141-5a9d-496f-b025-f514344b7129 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:35.065992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6ba19141-5a9d-496f-b025-f514344b7129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:35.137993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 499/991] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:35.160802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18fb2f-42d7-410d-8583-bde251b01189 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.243700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b19fd-5060-4622-b5a1-c6a306393c28 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.244103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b19fd-5060-4622-b5a1-c6a306393c28 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.244286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b19fd-5060-4622-b5a1-c6a306393c28 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.245686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 500/992] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:35.252052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.252342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.252624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.269131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:35.270357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:35.276820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18fb2f-42d7-410d-8583-bde251b01189 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.280571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:35.320940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.321707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.321996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.322274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.322941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.324314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:35.337320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:35.338586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:35.339564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 493/993] 10.222.0.79 () {56 vars in 1126 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:35.345047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.350410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 501/994] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:35.373647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.385726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:35.457827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3699fc3-1352-49b4-bdea-14e6335cfc22 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.497547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec5b6f-b359-4931-910c-e301b98ed878 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.502284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.502950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.503235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.503511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.504223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:35.504529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.506054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.511801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 494/995] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:35.517724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.541873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.543014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.543308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.543618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.544917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.547724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.551962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:35.562448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:35.564111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:35.565119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 502/996] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:35.569814 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.634353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a7a9c-8592-4685-8618-d44280c64e66 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.646115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:35.670932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.672683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.673003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.673284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.674002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:35.674314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.675694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.680905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 495/997] 10.222.0.79 () {56 vars in 1122 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:35.689858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5d60f-9d04-4622-9b28-e033cb5bc54e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.690042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.726729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.727398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.727687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.728078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.728937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:35.729264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.730636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.736069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 503/998] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:35.741674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.791331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:35.799053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d1bf-960f-45f6-a9ce-064b00efc6c4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.832694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed4dc0-ddad-4e76-a472-426ba13947fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.833382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4dc0-ddad-4e76-a472-426ba13947fd None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.833672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4dc0-ddad-4e76-a472-426ba13947fd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.833950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4dc0-ddad-4e76-a472-426ba13947fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.834688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed4dc0-ddad-4e76-a472-426ba13947fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.836054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed4dc0-ddad-4e76-a472-426ba13947fd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.847529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03995729-594c-4349-a97e-77371a48e1c2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:35.850986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 496/999] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:15:35.852070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4123d-b5eb-40a4-9b6c-d00f6fec5113 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.852361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4123d-b5eb-40a4-9b6c-d00f6fec5113 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.852638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4123d-b5eb-40a4-9b6c-d00f6fec5113 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.854056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 497/1000] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:35.859865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b9335-4598-4140-a820-30e03f6e858a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.860160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b9335-4598-4140-a820-30e03f6e858a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.860443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b9335-4598-4140-a820-30e03f6e858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.878371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-259b9335-4598-4140-a820-30e03f6e858a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:35.879647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-259b9335-4598-4140-a820-30e03f6e858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:35.899180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:35.907843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:35.908970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:35.909327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:35.909857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:35.911135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:35.911660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:35.913529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:35.920965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 504/1001] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:35.927196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:35.964090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 498/1002] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:35.990802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b9335-4598-4140-a820-30e03f6e858a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:36.056268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21910a-2d0e-4391-aac7-0282525e1e6a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:36.085498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:36.097612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:36.098490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.098916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.099383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.100432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] RBAC: Adding query filter params (interface=public, service_id=01d62ca9c93343f99f49c84eeb942937, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:36.100954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:36.102554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:36.102673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b9335-4598-4140-a820-30e03f6e858a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:36.113029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 505/1003] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/endpoints?service_id=01d62ca9c93343f99f49c84eeb942937&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:36.115039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:36.139072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:36.139866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.140147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.140525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.141325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:36.142694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:36.154538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:36.155717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:36.156754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 499/1004] 10.222.0.79 () {56 vars in 1105 bytes} [Wed Jul 20 19:15:35 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:36.161743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:36.183250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:36.231773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462f5c-dc1e-407b-a329-3a2c64516bb0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:36.268372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:36.269097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.269382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.269656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.270194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce3135-6fa6-438c-a4f8-e98160d056e0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:36.270337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:36.271707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:36.279085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 506/1005] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:36 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:15:36.322098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:36.322780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.323064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.323336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.324065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:36.324374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:36.327939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c13e7d-a8a1-4ead-99e5-6d4bcbbcb8c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:36.337686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 500/1006] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:36.346054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:36.460710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd1e27-20a6-4617-9f88-19a11f4b1838 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:36.483787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:36.496471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:36.497221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:36.497508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:36.497778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:36.498486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] RBAC: Adding query filter params (type=image) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:36.498788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:36.500851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fbbcd-7627-4580-9809-61fe57136f5a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:36.512184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 507/1007] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:37.950412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba6791-181f-43db-aa25-a783196caa6d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:37.950935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba6791-181f-43db-aa25-a783196caa6d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:37.951042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba6791-181f-43db-aa25-a783196caa6d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:37.952411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 501/1008] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:37.958326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:37.958788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:37.958788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:37.976333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:37.977577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:38.059582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 508/1009] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:38.083508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba6791-181f-43db-aa25-a783196caa6d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:38.101524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:38.193174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba6791-181f-43db-aa25-a783196caa6d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:38.227295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b973c-9a5c-4350-b1bf-552627863c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:38.228022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b973c-9a5c-4350-b1bf-552627863c9f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.228307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b973c-9a5c-4350-b1bf-552627863c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.228584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b973c-9a5c-4350-b1bf-552627863c9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.229797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b973c-9a5c-4350-b1bf-552627863c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:38.231228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b973c-9a5c-4350-b1bf-552627863c9f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:38.238201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 502/1010] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:38.239410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:38.341892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25955bc-d9e1-429f-870e-5f41a6902b6d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:38.358866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:38.379471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:38.380136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.380417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.380686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.381422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:38.382840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:38.389156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011393f-0889-4f1d-a8c6-a0ffddd56517 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.389449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011393f-0889-4f1d-a8c6-a0ffddd56517 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.389728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011393f-0889-4f1d-a8c6-a0ffddd56517 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.391951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 503/1011] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:38.396657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3271-7095-440f-81c0-f43eb2126b26 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.396975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3271-7095-440f-81c0-f43eb2126b26 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.397255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3271-7095-440f-81c0-f43eb2126b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.420914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 509/1012] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:38 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:15:38.423036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a1fe3271-7095-440f-81c0-f43eb2126b26 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:38.424367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a1fe3271-7095-440f-81c0-f43eb2126b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:38.503621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 504/1013] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:38.527644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:38.586119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e4e46-e823-4c75-8301-4ede3074cae9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.586119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e4e46-e823-4c75-8301-4ede3074cae9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.586119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e4e46-e823-4c75-8301-4ede3074cae9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.588143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 505/1014] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:38.594354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.594824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.595093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.614580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:38.616172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:38.624052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:38.645212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe02f51-8ec7-4ae6-98b2-555bd73f998a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:38.682459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:38.683267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.683557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.683841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.684500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:38.685921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:38.698391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:38.704772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:38.706196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 510/1015] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:38.707005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 506/1016] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:38.707083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:38.727823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:38.742927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:38.831562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd471b-1ccc-40ef-b719-b27398e76d08 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:38.840614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430055-6ad4-450a-8a2e-dadb57909437 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:38.867710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:38.868387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.868671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.868986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.869703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:38.870039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:38.871460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:38.876428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 511/1017] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:38.877623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:38.878490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:38.878946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:38.879415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:38.880307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:38.881860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:38.881969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:38.898360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 507/1018] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:15:38 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:15:39.000318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:39.008245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff344b5a-bfb0-42e8-8573-8b0d42c410d1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:39.055368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:39.056046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:39.056330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:39.056604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:39.057352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:39.057672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:39.059044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb73c3-9be6-4d27-973c-e0afb2308775 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:39.064210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 512/1019] 10.222.0.79 () {56 vars in 1112 bytes} [Wed Jul 20 19:15:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:39.070013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:39.073013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:39.185510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957401f8-0ae6-418a-b99e-68d0cf92d245 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:39.220862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e020f-bc4e-4c74-bebb-0a8bf1e5114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:39.221685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e020f-bc4e-4c74-bebb-0a8bf1e5114c None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:39.221977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e020f-bc4e-4c74-bebb-0a8bf1e5114c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:39.222257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e020f-bc4e-4c74-bebb-0a8bf1e5114c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:39.223009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e020f-bc4e-4c74-bebb-0a8bf1e5114c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:39.224388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e020f-bc4e-4c74-bebb-0a8bf1e5114c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:39.244991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 508/1020] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:40.544095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2213b8-bb2c-49e0-bd01-a3fc40e1669c None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:40.544693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2213b8-bb2c-49e0-bd01-a3fc40e1669c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:40.544693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2213b8-bb2c-49e0-bd01-a3fc40e1669c None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:40.546279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 513/1021] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:40.552334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:40.552827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:40.553069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:40.576779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:40.578226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:40.642680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:40.670575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 509/1022] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:40.699916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2213b8-bb2c-49e0-bd01-a3fc40e1669c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:40.811163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2213b8-bb2c-49e0-bd01-a3fc40e1669c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:40.845377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba75d53-2b5b-4e5e-b0e2-1b9ed0284772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:40.846185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75d53-2b5b-4e5e-b0e2-1b9ed0284772 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:40.846471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75d53-2b5b-4e5e-b0e2-1b9ed0284772 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:40.846748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75d53-2b5b-4e5e-b0e2-1b9ed0284772 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:40.847943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba75d53-2b5b-4e5e-b0e2-1b9ed0284772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:40.849419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba75d53-2b5b-4e5e-b0e2-1b9ed0284772 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:40.851240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 514/1023] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:40.856752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:40.967370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae683f1-00ac-4aa0-8722-534752693ec8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:40.990868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb285d89-2f31-438f-aa3d-92104d8fec73 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:40.991158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb285d89-2f31-438f-aa3d-92104d8fec73 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:40.991435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb285d89-2f31-438f-aa3d-92104d8fec73 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:40.992915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 515/1024] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:40.997582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:40.997867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:40.998140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.004534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.005243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.005530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.005812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.006532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.007963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.015445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:41.016755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:41.029985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 510/1025] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:15:40 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:41.037672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:41.098178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 516/1026] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:41.120761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.168834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:41.233956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca680-8cf6-4212-bf8f-fd9b801143bc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.270488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.271278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.271556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.271822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.272466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.273861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:41.285738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:41.286853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:41.287698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 511/1027] 10.222.0.79 () {56 vars in 1105 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:41.293171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.398046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4064c-851c-4182-83d3-ee539f71d6a9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.432708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.433418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.433701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.433974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.434686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] RBAC: Adding query filter params (name=image) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:41.434993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.436356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7e737-1252-4ad0-b5c3-07a8803c6a6b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.447515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 517/1028] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:41.448964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.449698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81bb9-0d99-4b08-bf79-a6f52c298a07 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.449992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81bb9-0d99-4b08-bf79-a6f52c298a07 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.450274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81bb9-0d99-4b08-bf79-a6f52c298a07 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.451757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 518/1029] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:41.457123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.457415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.457690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.475602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:41.476833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:41.491954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:41.553332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 519/1030] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:41.565823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e799c93-aebe-4328-9611-3376feddb81a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.569618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:41.577703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.606438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.607424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.607923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.608402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.609604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] RBAC: Adding query filter params (type=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:41.610153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.611753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.622166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 512/1031] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:41.623955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.689620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086486fc-ea2f-4cc4-931d-b31e0c381c58 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.727632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.728563 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.728992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.730152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.741462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2e248-1709-4ac6-bd4a-378d910365de None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.743607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:41.755104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:41.756093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:41.756997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:41.757664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 520/1032] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:41.764922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.778278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.778567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:41.778943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.779451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.779562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.780431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] RBAC: Adding query filter params (interface=public, service_id=97e59776d236494a93adbb2a3bb055c9, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:41.780781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.782825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.788434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 513/1033] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/endpoints?service_id=97e59776d236494a93adbb2a3bb055c9&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:41.793479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.827568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:41.884904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852df14-e495-4928-8d86-3b29502c7e63 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.915273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82297-04db-4989-91ca-e88bfaf0213a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:41.922796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.923456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.923742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.924022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.924767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:41.925107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.926571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.939736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 521/1034] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:41.945208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:41.962329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:41.963006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:41.963290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:41.963572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:41.964267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:41.965672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:41.984916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 514/1035] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:15:42.039891 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:42.079447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795a4f-2402-4494-9675-a991eadcdb11 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:42.126263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:42.127166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.127508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.127946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.142490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:42.144965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:42.163822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:42.164801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:42.165732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:42.165732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:42.165732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:42.165732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:42.165732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 522/1036] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:42.170989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:42.282677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:42.289764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b0fe3-1393-44ed-828f-3a50155e2b49 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:42.324339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:42.325040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.325334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.325602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.326333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:42.326645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:42.328045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:42.348912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 515/1037] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:42.351392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:42.466567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056bbed-01e0-4b45-a7ef-39de5f0618a0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:42.491935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:42.505276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:42.506091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.506376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.506650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.516094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:42.518012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:42.534337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:42.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:42.542260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 523/1038] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:15:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:42.542865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:42.652076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc08897-f2b3-4334-9912-ac09bff605ac None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:42.701951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:42.703159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.703646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.704113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.705662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:42.706192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:42.708576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:42.728778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 516/1039] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:15:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:42.734959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:42.858848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2817da-9bf0-4db1-87fc-98b9c85337ea None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:42.909494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8302bf5-1d41-4d65-9fc0-fbfce8b1d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:42.910338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8302bf5-1d41-4d65-9fc0-fbfce8b1d77b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.910705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8302bf5-1d41-4d65-9fc0-fbfce8b1d77b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.911087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8302bf5-1d41-4d65-9fc0-fbfce8b1d77b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.974019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.005186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8302bf5-1d41-4d65-9fc0-fbfce8b1d77b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.006677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8302bf5-1d41-4d65-9fc0-fbfce8b1d77b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.125860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 524/1040] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:43.132159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.151775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.196319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19445337-695f-4bdb-9c8c-56ce3ee27c31 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.196605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19445337-695f-4bdb-9c8c-56ce3ee27c31 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.196905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19445337-695f-4bdb-9c8c-56ce3ee27c31 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.198486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 525/1041] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:43.204676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.204997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.205274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.228447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:43.230406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:43.266833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92ff89-f80c-4a33-ba3b-000e06cd5c73 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.307684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 526/1042] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:43.311780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.312802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.313267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.313718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.325404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.327120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.331859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.346502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:43.349214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:43.350272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:43.351357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:43.351357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:43.351357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:43.351357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:43.357718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 517/1043] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:43.359016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.430061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.443183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcafc6-a527-4567-8421-e7121fecd7e7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.465617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297bf08-a104-4bb8-8da4-f42884701544 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.479380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.480483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.480966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.481412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.493196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:43.502405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:43.503580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:43.504572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:43.506837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 527/1044] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:43.510576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.511242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.511623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.511916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.512655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:43.512996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.513185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.514373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.527682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 518/1045] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:43.533530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.598130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.635847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c40cbc-1bbf-4591-b458-abd21583ee75 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.649194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34a6cd-252a-40f1-9ddd-4a347efdeef2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.674393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.675334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.675624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.675903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.676661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:43.677007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.678390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.681521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.689830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 528/1046] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:43.694534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.698214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.699264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.700193 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.701725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.710633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.711913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:43.727339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:43.728267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:43.729896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 519/1047] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:43.734904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.820667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0f9c0-1d01-4ebf-99db-eaee0af30c91 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.853423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db631ff5-87f6-41f3-bfb3-c47b62386918 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:43.857093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.857983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.858268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.858542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] PATH_INFO: `/v3/users/e32b556b021349b4bd262a6b156e87e2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.860845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:43.873609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] RBAC: Authorizing `identity:get_user(user_id=e32b556b021349b4bd262a6b156e87e2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.876057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.878231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 529/1048] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/users/e32b556b021349b4bd262a6b156e87e2 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:43.883827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.892676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:43.893721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:43.894188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:43.894672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:43.895769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:43.896259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:43.898070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:43.910788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 520/1049] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:43.916869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:43.970109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.000977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4d580-0c1d-4a3b-8f9b-22fb885d7ecb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.028733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99908ead-5059-495e-8aa4-aa23ceeb8d2b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.038937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.039603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.039890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.040163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.066520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.067478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.067994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.068473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.069345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] RBAC: Adding query filter params (role.id=2aab0e07f8ec44a7a5d961b8e00c62ca, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=9918e21a598f4668b5fde7b173b647ac) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:44.069663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.071103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9e54-c4a3-4def-9666-e815f4813bfe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.089087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 521/1050] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/role_assignments?user.id=9918e21a598f4668b5fde7b173b647ac&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=2aab0e07f8ec44a7a5d961b8e00c62ca => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:44.110280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.114407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0a0b7-9342-4307-ada2-86ac1f62aa04 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.114702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0a0b7-9342-4307-ada2-86ac1f62aa04 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.114992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0a0b7-9342-4307-ada2-86ac1f62aa04 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.116631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 522/1051] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:44.122029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.122200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.122396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.122659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.123503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.142278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:44.143734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:44.222984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 523/1052] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:44.241966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 530/1053] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:44.245772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.247756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.290408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.342526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.351527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febde53-4ebe-4aca-a57b-58b6adf22f5c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.351908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fcaab-cbed-4392-b78e-d536877a222d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.388204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.388446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.389037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.389319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.389593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.390242 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.390364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.390928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.391399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.391646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.393006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] RBAC: Authorizing `identity:get_project(project_id=68bf8342d732486db04b95bcaa608c9e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.394651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.401424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 531/1054] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:15:44 2022] GET /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:44.402689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:44.403846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:44.404832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:44.405758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 524/1055] 10.222.0.79 () {56 vars in 1105 bytes} [Wed Jul 20 19:15:44 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:44.411456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.512867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c72437-d22d-4f79-8120-17e9f7958522 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.523040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71265c5b-bdb1-45ee-8295-35a74aa41996 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.526030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.553228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.554180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.554465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.554744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.555581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=68bf8342d732486db04b95bcaa608c9e, user.id=e32b556b021349b4bd262a6b156e87e2) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:44.555906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.557358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.563541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 532/1056] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:44 2022] GET /identity/v3/role_assignments?user.id=e32b556b021349b4bd262a6b156e87e2&scope.project.id=68bf8342d732486db04b95bcaa608c9e&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:44.567191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.567867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.568149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.568429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.569202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:44.569517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.570999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.577979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 525/1057] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:44.583774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.613912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:44.705916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36b6b-efbd-4487-8498-4963df5e6fe4 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.745872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.746545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.746832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.747112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.747825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:44.748135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.750084 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533fb96-1ebf-4422-aca6-6afb7f65cac9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.755568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 533/1058] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:44.761488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:44.868918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8ec21-07dc-4004-a170-955fa20c6209 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:44.903589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5495e-43aa-46ab-9461-1b28c629653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:44.904257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5495e-43aa-46ab-9461-1b28c629653f None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:44.904542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5495e-43aa-46ab-9461-1b28c629653f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:44.904817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5495e-43aa-46ab-9461-1b28c629653f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:44.905580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5495e-43aa-46ab-9461-1b28c629653f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:44.907308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5495e-43aa-46ab-9461-1b28c629653f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:44.931691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 526/1059] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:15:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:46.194161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ecf9-020f-41d8-9b94-762702d2764e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.194725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ecf9-020f-41d8-9b94-762702d2764e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.194725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ecf9-020f-41d8-9b94-762702d2764e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.196137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 534/1060] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:46.201390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.201798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.201985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.219605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:46.220802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:46.239241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:46.297035 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 527/1061] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:46.319901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ecf9-020f-41d8-9b94-762702d2764e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:46.413066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:46.427287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ecf9-020f-41d8-9b94-762702d2764e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:46.461627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d776-e893-4e6d-97e6-b6ec67a1b9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:46.462360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d776-e893-4e6d-97e6-b6ec67a1b9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.462645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d776-e893-4e6d-97e6-b6ec67a1b9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.462920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d776-e893-4e6d-97e6-b6ec67a1b9d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.473708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d776-e893-4e6d-97e6-b6ec67a1b9d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3776d776-e893-4e6d-97e6-b6ec67a1b9d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:46.481933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:46.482870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:46.483804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:46.488692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 535/1062] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Jul 20 19:15:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:46.490065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:46.592074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad00a7a-2996-46fd-85e2-398e811cbf2a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:46.626324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae831d4-c52b-4903-969a-79937ff1d40e None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.626633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae831d4-c52b-4903-969a-79937ff1d40e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.626913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae831d4-c52b-4903-969a-79937ff1d40e None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.628235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:46.628380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 536/1063] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:46.629416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.629901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.630357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.631299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:46.631883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:46.633273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.633565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.633846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.633950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:46.644385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 528/1064] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:15:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:46.650016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:46.655018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:46.656850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:46.667499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:46.739984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 537/1065] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:46.762158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:46.770958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5a37b-8c9a-467f-a80b-300579236b2e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:46.816210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:46.817338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.817628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.817906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.831649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:46.833141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:46.847495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:46.851939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:46.852827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:46.853782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:46.853782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:46.853782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:46.853782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:46.853782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 529/1066] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:46.859596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:46.874759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09808f-f568-4b20-bafa-fe8505ad0fc1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:46.912322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:46.913157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:46.913507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:46.913793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:46.925639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:46.931173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:46.933846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:46.934988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:46.938269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 538/1067] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:46.941873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:46.979355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a669df-3036-4c42-8d59-9a2ed9340251 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.026538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.027131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.027681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.027961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.028697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:47.029039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.030417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5a2fb-0b2b-4796-87e3-3b70bc050454 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.043896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 530/1068] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:47.045202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9a260-db8d-4e33-97f4-4f54f37c1595 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.045544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9a260-db8d-4e33-97f4-4f54f37c1595 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.045769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9a260-db8d-4e33-97f4-4f54f37c1595 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.048926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 531/1069] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:47.050382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9a260-db8d-4e33-97f4-4f54f37c1595 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.051666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfb832-0413-45f4-a6d9-304a82f9ce43 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.086026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.086698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.086987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.087379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.088014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:47.088324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.089722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711402b-7817-4bcb-b450-0133122fcae6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.095717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.100783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 539/1070] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:47.102024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.102321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.102596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.118355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:47.119576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:47.162773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9a260-db8d-4e33-97f4-4f54f37c1595 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.189266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.190897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 540/1071] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:47.191758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.210523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.211869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.212372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.212866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.226360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.227937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:47.243194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:47.244210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:47.245231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 532/1072] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:47.245948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.301756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53c6a8-6654-455d-8f3d-82fd3e819fa0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.330294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.336805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.337718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.338002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.338272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] PATH_INFO: `/v3/users/e32b556b021349b4bd262a6b156e87e2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.339709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] RBAC: Authorizing `identity:get_user(user_id=e32b556b021349b4bd262a6b156e87e2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.341175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.342818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 541/1073] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/users/e32b556b021349b4bd262a6b156e87e2 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:47.344126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.346233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33942e45-3a75-4833-a37b-aa0f61023915 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.383117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.383773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.384057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.384336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.436938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.448554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe09fa-f455-4cff-b52b-ab05c3593ac9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.454534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.456112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.486835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.487724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.487839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.488038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.488902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:47.489215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.490661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.508089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 542/1074] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:47.508959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.565021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 533/1075] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:47.565785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.575468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.620583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057a434-0a83-4378-baf4-c2eea6d13e12 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.652158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.656323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.657219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.657502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.657775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.658948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] RBAC: Authorizing `identity:get_project(project_id=68bf8342d732486db04b95bcaa608c9e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.660371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.662105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 543/1076] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:47.662857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.670317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e08c-551f-405c-b4c2-8a987f60b331 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.710923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.711733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.712010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.712285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.722956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.724424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:47.742320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:47.743308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:47.744219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:47.744219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:47.744219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:47.744219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:47.744219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 534/1077] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:47.744507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.768169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e4bb-3899-4c98-a62b-8e72dc4a78e1 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.792362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.804869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.805819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.806110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.806390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.817831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.819219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:47.836966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:47.838034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:47.838928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:47.838928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:47.838928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:47.838928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:47.838928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 544/1078] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:47.839216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.860571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a3f18-dbfe-48a5-8eef-607ddb57f466 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.895728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:47.896607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.896906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.897183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e/users/e32b556b021349b4bd262a6b156e87e2/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.899138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] RBAC: Authorizing `identity:create_grant(project_id=68bf8342d732486db04b95bcaa608c9e, user_id=e32b556b021349b4bd262a6b156e87e2, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:47.900555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:47.903782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.916129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 535/1079] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:47 2022] PUT /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e/users/e32b556b021349b4bd262a6b156e87e2/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:15:47.916960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:47.955586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c432f8d-fc21-423b-9c28-63e58ffe8770 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:47.998871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:48.000136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.000422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.000697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.001493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:48.001809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:48.003244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:48.016991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 545/1080] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:48.022265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:48.045880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556d435-5e20-4793-af2f-a2d1e28f1e48 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:48.064484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:48.090982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:48.091575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.091922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.092207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.092990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:48.093862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:48.095383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:48.112934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 536/1081] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:48.142327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb931527-e4f5-4121-bcf3-f8ddc95eb901 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:48.183576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:48.184390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.184708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.185030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.194743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:48.196554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:48.213283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:48.214686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:48.215609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 546/1082] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:48.221560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:48.295576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:48.342105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424c224-58d6-4034-ac1f-c5f8b183f6e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:48.378361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:48.379225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.379718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.380189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.381043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:48.381356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:48.382782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca05fb-d501-4da3-81a2-5d0070663504 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:48.396424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 537/1083] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:48.402909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:48.502383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:48.509126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9f76-79e3-4a4a-bb65-e82a8aea7b3c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:48.548711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37374a3a-6e9f-4463-9eb7-fa304216e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:48.549637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37374a3a-6e9f-4463-9eb7-fa304216e44f None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.549917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37374a3a-6e9f-4463-9eb7-fa304216e44f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.550190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37374a3a-6e9f-4463-9eb7-fa304216e44f None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/9918e21a598f4668b5fde7b173b647ac/roles/2aab0e07f8ec44a7a5d961b8e00c62ca` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.552480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37374a3a-6e9f-4463-9eb7-fa304216e44f None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=9918e21a598f4668b5fde7b173b647ac, role_id=2aab0e07f8ec44a7a5d961b8e00c62ca)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:48.554963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37374a3a-6e9f-4463-9eb7-fa304216e44f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:48.578421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 547/1084] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:48 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/9918e21a598f4668b5fde7b173b647ac/roles/2aab0e07f8ec44a7a5d961b8e00c62ca => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:15:49.995495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bbdde-3224-410f-ad75-f9fac61a3e6a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:49.996001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bbdde-3224-410f-ad75-f9fac61a3e6a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:49.996001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bbdde-3224-410f-ad75-f9fac61a3e6a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:49.997392 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 538/1085] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:50.003521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.003950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.004084 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.021496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:50.022680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:50.089195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 548/1086] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:50.113322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bbdde-3224-410f-ad75-f9fac61a3e6a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.215113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bbdde-3224-410f-ad75-f9fac61a3e6a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:50.235219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:50.254350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:50.255069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.255355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.255630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.266146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:50.273821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:50.274833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:50.275866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 539/1087] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:50.281336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.368218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:50.384534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40bbef-bdfe-4436-a0ad-cd2b16d677e0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:50.418271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:50.418957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.419236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.419507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.420367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:50.420762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:50.422700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:50.430711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 549/1088] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:50.436633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.537904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44ee66-b8a9-4f8c-a2d9-605c8971ef66 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:50.572256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7745a-d9f4-44f7-8c7b-f43611562bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:50.573166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7745a-d9f4-44f7-8c7b-f43611562bcf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.573452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7745a-d9f4-44f7-8c7b-f43611562bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.573729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7745a-d9f4-44f7-8c7b-f43611562bcf None admin] PATH_INFO: `/v3/users/e32b556b021349b4bd262a6b156e87e2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.574909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7745a-d9f4-44f7-8c7b-f43611562bcf None admin] RBAC: Authorizing `identity:get_user(user_id=e32b556b021349b4bd262a6b156e87e2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:50.576344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7745a-d9f4-44f7-8c7b-f43611562bcf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:50.578211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 540/1089] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/users/e32b556b021349b4bd262a6b156e87e2 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:50.583890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.597287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59419045-46f9-42e9-b8a9-db2e338224ea None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.597584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59419045-46f9-42e9-b8a9-db2e338224ea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.597868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59419045-46f9-42e9-b8a9-db2e338224ea None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.599234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 541/1090] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:15:50.604089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.604415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.604702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.621909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:50.623118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:50.664192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:50.691646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2650f-0544-4857-90dd-f759ac43271f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:50.693692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 542/1091] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:50.717484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.725949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:50.726610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.726899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.727167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.745462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:50.797574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:50.799104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:50.823395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c27a9-1029-45fd-a655-b116f7eca68e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:50.858339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:50.860014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:50.860327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:50.860606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:50.870705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:15:50.879913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:50.880972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:50.881831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 543/1092] 10.222.0.79 () {56 vars in 1120 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:50.887058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.899504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 550/1093] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:50.905704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:50.922775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:50.954463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:50.993258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482346-048d-4c51-a60c-76e54645b330 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.012207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af331332-6cbe-45de-868b-9502918d4e26 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.029880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.030555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.030903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.031176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.032027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:51.032329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.033728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.041826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 544/1094] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:51.047883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:51.048654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.049544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.049832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.050110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.051332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] RBAC: Authorizing `identity:get_project(project_id=68bf8342d732486db04b95bcaa608c9e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.052781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.054523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 551/1095] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:15:50 2022] GET /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:51.060431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:51.142580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:51.149080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0abdc1-65ca-4ca5-825b-29e5ca53e82b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.161371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4467-52f9-4f1e-aa4d-c67b986d6c14 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.178109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:51.183308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.184108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.184391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.184668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.196632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.197037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.198249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.198249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.198439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.198527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.199070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=68bf8342d732486db04b95bcaa608c9e, user.id=e32b556b021349b4bd262a6b156e87e2) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:51.199382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.200786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.207483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 552/1096] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:51 2022] GET /identity/v3/role_assignments?user.id=e32b556b021349b4bd262a6b156e87e2&scope.project.id=68bf8342d732486db04b95bcaa608c9e&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:51.216250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:51.217617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:51.218924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:51.218924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:51.218924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:51.218924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:51.218924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 545/1097] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:15:51.223663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:51.330861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab15161-e6c7-4d77-9e02-35e2e5442d9c None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.367034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.367708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.367993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.369713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.369713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:51.369713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.370680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.387631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 553/1098] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:51.394413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:51.496433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff279-7b66-44d1-b6d1-580b7f5ad5bd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.531297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.532092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.532376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.532650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.540248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.541725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:15:51.556181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:51.557324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:15:51.558310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 546/1099] 10.222.0.79 () {56 vars in 1093 bytes} [Wed Jul 20 19:15:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:51.564132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:51.606010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:51.665049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5fe01-dfd0-49ea-9374-524b2c03ac1b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.703854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.704528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.704806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.705118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.705952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:51.706258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.707673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.724786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 554/1100] 10.222.0.79 () {56 vars in 1133 bytes} [Wed Jul 20 19:15:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:51.731208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:51.742571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:51.835407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca3294-9a96-46a8-a86e-e6f31de083cd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:51.869190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:51.869763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:51.870052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:51.870331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:51.938892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:51.940347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:51.950264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.041289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 547/1101] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:52.048450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.144167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.148080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc0fd-aa5c-4f9e-82cf-425770a644b7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:52.182021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:52.182806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.183088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.183360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.193841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:52.195271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:15:52.213042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:52.214135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:52.215037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:15:52.215037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:15:52.215037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:15:52.215037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:52.215037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 555/1102] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:52.220144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.321480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eb0e1-5fe3-4102-a3a8-33eac53d6ac8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:52.331233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.356610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:52.357336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.357623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.357903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.359174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:52.359497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:52.360982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:52.373249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 548/1103] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:52.379931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.483220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995bff2-6dab-423a-a51f-bba4e19c8f9a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:52.517681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:52.518468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.518751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.519027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.525663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:52.527103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:15:52.541916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:15:52.542922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:15:52.543822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 556/1104] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:15:52.549015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.649264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d5b95-86c2-4364-9406-17faf0161756 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:52.682106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:52.682753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.683036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.683308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.684091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:52.684398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:52.685846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ac79a-000f-4af7-9829-368676c5b73a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:52.700753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 549/1105] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:52.706912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.720794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.811787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b024-cad9-4c41-b31b-e33b9e303254 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:52.850989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:52.851648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.851982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.852207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.853081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] RBAC: Adding query filter params (role.id=2aab0e07f8ec44a7a5d961b8e00c62ca, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=9918e21a598f4668b5fde7b173b647ac) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:52.853395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:52.854834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacf3be-cb73-4e8c-b309-b06297c51d52 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:52.861575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 557/1106] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/role_assignments?user.id=9918e21a598f4668b5fde7b173b647ac&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=2aab0e07f8ec44a7a5d961b8e00c62ca => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:53.310367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb574f-70eb-4f38-9089-07203cc83e70 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:53.310884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb574f-70eb-4f38-9089-07203cc83e70 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:53.311004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb574f-70eb-4f38-9089-07203cc83e70 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:53.312927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 550/1107] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:53.318786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:53.319229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:53.319427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:53.337156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:53.338359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:53.410995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 558/1108] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:53.435422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb574f-70eb-4f38-9089-07203cc83e70 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:53.467040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:53.553530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb574f-70eb-4f38-9089-07203cc83e70 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:53.589656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a9a2-ca00-41fc-8a86-122d34358ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:53.590377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a9a2-ca00-41fc-8a86-122d34358ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:53.590665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a9a2-ca00-41fc-8a86-122d34358ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:53.590945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a9a2-ca00-41fc-8a86-122d34358ad5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:53.592164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a9a2-ca00-41fc-8a86-122d34358ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:53.594258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a9a2-ca00-41fc-8a86-122d34358ad5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:53.595921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 551/1109] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:15:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:53.602157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:53.607010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:53.723014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d402e5-27ec-486a-a13e-93543c050df7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:53.759090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04482ee2-d469-47e8-af00-ae4caf83a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:53.759772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04482ee2-d469-47e8-af00-ae4caf83a5ae None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:53.760057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04482ee2-d469-47e8-af00-ae4caf83a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:53.760336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04482ee2-d469-47e8-af00-ae4caf83a5ae None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:53.761114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04482ee2-d469-47e8-af00-ae4caf83a5ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:53.762574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04482ee2-d469-47e8-af00-ae4caf83a5ae None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:53.783218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 559/1110] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:15:53 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:15:55.887920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c092b-ef1a-44e4-a37a-665bd5078997 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:55.888500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c092b-ef1a-44e4-a37a-665bd5078997 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:55.888500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c092b-ef1a-44e4-a37a-665bd5078997 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:55.889984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 552/1111] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:55.896096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:55.896531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:55.896653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:55.913524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:55.914704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:55.970638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:55.982854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 560/1112] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:15:56.007080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c092b-ef1a-44e4-a37a-665bd5078997 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:56.102694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.110870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c092b-ef1a-44e4-a37a-665bd5078997 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.145348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.146205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.146492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.146773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] PATH_INFO: `/v3/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.148251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] RBAC: Authorizing `identity:get_role(role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.149641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.151030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 553/1113] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:56 2022] GET /identity/v3/roles/a0ef0911aac646be8916503dba5a9c3f => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:56.157556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:56.257571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e25ec-15c3-4a5b-80ab-6ac88811d167 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.290942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.291804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.292085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.292353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] PATH_INFO: `/v3/users/dfedff25b22b46ed8a22a40c62bb3d0e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.307123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] RBAC: Authorizing `identity:get_user(user_id=dfedff25b22b46ed8a22a40c62bb3d0e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.308585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.310373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 561/1114] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:56 2022] GET /identity/v3/users/dfedff25b22b46ed8a22a40c62bb3d0e => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:56.317279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:56.428669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b19a-5921-42d6-9a4b-91004b7bba32 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.463176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.463819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.464101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.464374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.493558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.534731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.536177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.632386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 554/1115] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:15:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:15:56.638295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:56.687928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.739902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f26b4-3cf0-45ca-af59-3324ad146050 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.785966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f996ddc-af41-4baf-98cc-1d66d5dbf51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.786747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f996ddc-af41-4baf-98cc-1d66d5dbf51d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.787029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f996ddc-af41-4baf-98cc-1d66d5dbf51d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.787304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f996ddc-af41-4baf-98cc-1d66d5dbf51d None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.788475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f996ddc-af41-4baf-98cc-1d66d5dbf51d None admin] RBAC: Authorizing `identity:get_project(project_id=68bf8342d732486db04b95bcaa608c9e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.789964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f996ddc-af41-4baf-98cc-1d66d5dbf51d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.792018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 562/1116] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:15:56 2022] GET /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:56.798950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:56.860591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.899036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6cd51-9193-4af2-8556-a70fd5687d76 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.933925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.934562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.934845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.935118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.935936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] RBAC: Adding query filter params (role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=68bf8342d732486db04b95bcaa608c9e, user.id=dfedff25b22b46ed8a22a40c62bb3d0e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:15:56.936246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.937819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c03517-e7ad-44a1-94eb-e0c7c3ba20a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.943071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 555/1117] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:15:56 2022] GET /identity/v3/role_assignments?user.id=dfedff25b22b46ed8a22a40c62bb3d0e&scope.project.id=68bf8342d732486db04b95bcaa608c9e&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:58.988240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6c5e-f7e8-4f62-9974-d31d5d2aee58 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:58.988760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6c5e-f7e8-4f62-9974-d31d5d2aee58 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:58.988760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6c5e-f7e8-4f62-9974-d31d5d2aee58 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:58.990222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 563/1118] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:58.995394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:58.995809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:58.995963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.013016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:59.014202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:59.083522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 556/1119] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:15:59.105793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6c5e-f7e8-4f62-9974-d31d5d2aee58 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:59.208466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6c5e-f7e8-4f62-9974-d31d5d2aee58 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:59.225105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:59.243571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:59.244361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.244645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.244950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] PATH_INFO: `/v3/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.246413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] RBAC: Authorizing `identity:get_role(role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:59.247802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:59.250055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 564/1120] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:59 2022] GET /identity/v3/roles/a0ef0911aac646be8916503dba5a9c3f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:59.259567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:59.317431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:59.362226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f0724-3641-4668-8d68-cb86b1e087e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:59.395993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:59.396860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.397169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.397441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] PATH_INFO: `/v3/users/dfedff25b22b46ed8a22a40c62bb3d0e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.398604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] RBAC: Authorizing `identity:get_user(user_id=dfedff25b22b46ed8a22a40c62bb3d0e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:59.400034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:59.401685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 557/1121] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:15:59 2022] GET /identity/v3/users/dfedff25b22b46ed8a22a40c62bb3d0e => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:59.408011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:59.507994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d67de-f050-4b7c-a4da-13a3a631bda2 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:59.545181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750378a-00fb-4f4d-9818-d4166f68ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:59.546576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750378a-00fb-4f4d-9818-d4166f68ba55 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.546872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750378a-00fb-4f4d-9818-d4166f68ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.547147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750378a-00fb-4f4d-9818-d4166f68ba55 None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.548327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750378a-00fb-4f4d-9818-d4166f68ba55 None admin] RBAC: Authorizing `identity:get_project(project_id=68bf8342d732486db04b95bcaa608c9e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:59.549819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750378a-00fb-4f4d-9818-d4166f68ba55 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:59.551532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 565/1122] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:15:59 2022] GET /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:15:59.557929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:59.658311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acd70c-19cd-4636-b166-2e6242ea4ece None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:59.683952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:59.693174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788aa3b-061a-440d-b0fe-69b653c7d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:59.694064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788aa3b-061a-440d-b0fe-69b653c7d06b None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.694349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788aa3b-061a-440d-b0fe-69b653c7d06b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.694683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788aa3b-061a-440d-b0fe-69b653c7d06b None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e/users/dfedff25b22b46ed8a22a40c62bb3d0e/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.697045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788aa3b-061a-440d-b0fe-69b653c7d06b None admin] RBAC: Authorizing `identity:create_grant(project_id=68bf8342d732486db04b95bcaa608c9e, user_id=dfedff25b22b46ed8a22a40c62bb3d0e, role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:59.698482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788aa3b-061a-440d-b0fe-69b653c7d06b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:59.712224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 558/1123] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:15:59 2022] PUT /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e/users/dfedff25b22b46ed8a22a40c62bb3d0e/roles/a0ef0911aac646be8916503dba5a9c3f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:16:01.750295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663ee60-6155-4244-92a5-0c45662ddb6a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.750842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663ee60-6155-4244-92a5-0c45662ddb6a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.750842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663ee60-6155-4244-92a5-0c45662ddb6a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.752227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 566/1124] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:01.757578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.757974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.758135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.775772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:01.777009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:01.846388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 559/1125] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:01.869418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663ee60-6155-4244-92a5-0c45662ddb6a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:01.925095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:01.974267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663ee60-6155-4244-92a5-0c45662ddb6a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:02.007765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:02.008645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:02.009136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:02.009418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] PATH_INFO: `/v3/roles/a0ef0911aac646be8916503dba5a9c3f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:02.010734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] RBAC: Authorizing `identity:get_role(role_id=a0ef0911aac646be8916503dba5a9c3f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:02.012556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:02.013454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 567/1126] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:01 2022] GET /identity/v3/roles/a0ef0911aac646be8916503dba5a9c3f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:02.020062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:02.119531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437012a0-2ee8-40a9-85e0-c64ea58b17a9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:02.136322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:02.156540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:02.157486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:02.157776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:02.158048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] PATH_INFO: `/v3/users/dfedff25b22b46ed8a22a40c62bb3d0e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:02.159254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] RBAC: Authorizing `identity:get_user(user_id=dfedff25b22b46ed8a22a40c62bb3d0e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:02.160685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:02.162361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 560/1127] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:02 2022] GET /identity/v3/users/dfedff25b22b46ed8a22a40c62bb3d0e => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:02.168193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:02.266506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7d1c-145a-4533-b36f-8a4f4d659d2d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:02.286962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:02.301727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:02.302375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:02.302656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:02.302933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:02.370259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:02.371710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:02.469656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 568/1128] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:16:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:16:02.475697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:02.583272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4565efb-7fdd-4217-932e-6155a51fa477 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:02.618107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d520ff1-027d-401d-8c06-3c9981d515d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:02.618964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d520ff1-027d-401d-8c06-3c9981d515d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:02.619246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d520ff1-027d-401d-8c06-3c9981d515d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:02.619517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d520ff1-027d-401d-8c06-3c9981d515d6 None admin] PATH_INFO: `/v3/projects/68bf8342d732486db04b95bcaa608c9e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:02.620697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d520ff1-027d-401d-8c06-3c9981d515d6 None admin] RBAC: Authorizing `identity:get_project(project_id=68bf8342d732486db04b95bcaa608c9e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:02.622165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d520ff1-027d-401d-8c06-3c9981d515d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:02.623885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 561/1129] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:02 2022] GET /identity/v3/projects/68bf8342d732486db04b95bcaa608c9e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:02.630733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:02.653119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:02.733055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394313ad-44f4-4854-ba62-561f5493d067 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:02.769251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:02.769966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:02.770183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:02.770466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:02.771302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] RBAC: Adding query filter params (role.id=a0ef0911aac646be8916503dba5a9c3f, scope.project.id=68bf8342d732486db04b95bcaa608c9e, user.id=dfedff25b22b46ed8a22a40c62bb3d0e) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:02.771616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:02.773070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefba64-5d46-48e8-94db-69cf07b8de0e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:02.779292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 569/1130] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:16:02 2022] GET /identity/v3/role_assignments?user.id=dfedff25b22b46ed8a22a40c62bb3d0e&scope.project.id=68bf8342d732486db04b95bcaa608c9e&role.id=a0ef0911aac646be8916503dba5a9c3f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:04.863122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032cc25-be84-40ac-aa7b-b0cdd62ad672 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:04.863657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032cc25-be84-40ac-aa7b-b0cdd62ad672 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:04.863657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032cc25-be84-40ac-aa7b-b0cdd62ad672 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:04.865078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 562/1131] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:04.870253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:04.870681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:04.870822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:04.887402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:04.888608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:04.960753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 570/1132] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:04.984550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032cc25-be84-40ac-aa7b-b0cdd62ad672 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:05.040157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:05.089009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032cc25-be84-40ac-aa7b-b0cdd62ad672 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:05.123333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95cd2a-8f79-4006-abea-c6bbc5a33ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:05.124297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95cd2a-8f79-4006-abea-c6bbc5a33ebe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:05.124591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95cd2a-8f79-4006-abea-c6bbc5a33ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:05.124863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95cd2a-8f79-4006-abea-c6bbc5a33ebe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:05.126072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95cd2a-8f79-4006-abea-c6bbc5a33ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:05.127507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95cd2a-8f79-4006-abea-c6bbc5a33ebe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:05.129169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 563/1133] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:16:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:05.135093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:05.150977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:05.245574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9074e-aa9a-4e0a-bf64-185e3ef66a35 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:05.281935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9f4b7-73a1-4ea3-ba59-435072dde8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:05.282594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9f4b7-73a1-4ea3-ba59-435072dde8af None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:05.282877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9f4b7-73a1-4ea3-ba59-435072dde8af None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:05.283151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9f4b7-73a1-4ea3-ba59-435072dde8af None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:05.283864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9f4b7-73a1-4ea3-ba59-435072dde8af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:05.285330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9f4b7-73a1-4ea3-ba59-435072dde8af None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:05.311025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 571/1134] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:16:05 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:16:07.416743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7564f4-88bb-414d-975b-141b40ff304f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:07.417357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7564f4-88bb-414d-975b-141b40ff304f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:07.417357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7564f4-88bb-414d-975b-141b40ff304f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:07.418714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 564/1135] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:07.424607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:07.425075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:07.425192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:07.442245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:07.443422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:07.502261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:07.511179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 572/1136] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:07.536901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7564f4-88bb-414d-975b-141b40ff304f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:07.639993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7564f4-88bb-414d-975b-141b40ff304f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:07.660110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:07.674661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5b77-a582-4e6d-a484-a79ef8543c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:07.675388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5b77-a582-4e6d-a484-a79ef8543c1b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:07.675672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5b77-a582-4e6d-a484-a79ef8543c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:07.675944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5b77-a582-4e6d-a484-a79ef8543c1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:07.677127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5b77-a582-4e6d-a484-a79ef8543c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:07.678556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5b77-a582-4e6d-a484-a79ef8543c1b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:07.680186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 565/1137] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:16:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:07.686082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:07.792323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9c09-e26b-4fa5-8fa9-f6d2d8efccb4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:07.826815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd4bc-407e-41de-b35c-09dd2bf739a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:07.827529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd4bc-407e-41de-b35c-09dd2bf739a5 None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:07.827889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd4bc-407e-41de-b35c-09dd2bf739a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:07.828168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd4bc-407e-41de-b35c-09dd2bf739a5 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:07.828946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd4bc-407e-41de-b35c-09dd2bf739a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:07.830399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd4bc-407e-41de-b35c-09dd2bf739a5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:07.851129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 573/1138] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:16:07 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:16:09.947806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f4bd8-171c-492b-bc1c-6bb7203716c5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:09.948366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f4bd8-171c-492b-bc1c-6bb7203716c5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:09.948366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f4bd8-171c-492b-bc1c-6bb7203716c5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:09.950508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 566/1139] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:09.956227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:09.956645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:09.956752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:09.973522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:09.974720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:10.028934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.046155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 574/1140] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:10.071791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f4bd8-171c-492b-bc1c-6bb7203716c5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:10.174971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f4bd8-171c-492b-bc1c-6bb7203716c5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:10.209496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:10.210204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.210487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.210764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.222601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:16:10.230222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:10.231373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:16:10.232487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 567/1141] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:10.237843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:10.336958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618012f-a4e3-43a0-8c78-5c1ddcefbb14 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:10.370698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:10.371364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.371642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.371913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.372656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:10.372991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:10.375174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:10.383372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 575/1142] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:10.389168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:10.439885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.491707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587c21e-5bb6-4d1d-86e2-ff8d316b6bf6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:10.525851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:10.526717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.527002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.527275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] PATH_INFO: `/v3/users/bbfcd1ec15234f6e9226ea9ddebf96d0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.541373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfcd1ec15234f6e9226ea9ddebf96d0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:10.542821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:10.544557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 568/1143] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:10 2022] GET /identity/v3/users/bbfcd1ec15234f6e9226ea9ddebf96d0 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:10.551073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:10.597010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.652203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273196cb-1215-4ec1-a187-089a3000b937 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:10.686081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:10.687323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.687610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.687898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.757401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:10.758898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:10.803270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.857566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 576/1144] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:16:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:16:10.864199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:10.965529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a278a05-b48f-442c-82ac-4c565036e120 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:10.973816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:11.001727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38db58-aa3e-45bf-9112-e96a051bc621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:11.002579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38db58-aa3e-45bf-9112-e96a051bc621 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:11.002865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38db58-aa3e-45bf-9112-e96a051bc621 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:11.003143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38db58-aa3e-45bf-9112-e96a051bc621 None admin] PATH_INFO: `/v3/projects/29e6b2b8ed104be5a5cc461a63936813` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:11.004373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38db58-aa3e-45bf-9112-e96a051bc621 None admin] RBAC: Authorizing `identity:get_project(project_id=29e6b2b8ed104be5a5cc461a63936813)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:11.005825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38db58-aa3e-45bf-9112-e96a051bc621 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:11.007571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 569/1145] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:10 2022] GET /identity/v3/projects/29e6b2b8ed104be5a5cc461a63936813 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:11.014026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:11.118252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81141f6-5b7f-4ece-8cfd-0e4ad6a58c5e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:11.153388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:11.154045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:11.154329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:11.154679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:11.155832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=29e6b2b8ed104be5a5cc461a63936813, user.id=bbfcd1ec15234f6e9226ea9ddebf96d0) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:11.156302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:11.158061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cfcc5-9a6e-490b-8467-bd0596ce1815 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:11.164032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 577/1146] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:16:11 2022] GET /identity/v3/role_assignments?user.id=bbfcd1ec15234f6e9226ea9ddebf96d0&scope.project.id=29e6b2b8ed104be5a5cc461a63936813&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:13.197405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cfee-5597-442c-8862-14a41431a1b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.197967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cfee-5597-442c-8862-14a41431a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.197967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9cfee-5597-442c-8862-14a41431a1b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.199336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 570/1147] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:13.205389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.205828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.205938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.222597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:13.223803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:13.225825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:13.291168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 578/1148] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:13.314950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cfee-5597-442c-8862-14a41431a1b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:13.421595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9cfee-5597-442c-8862-14a41431a1b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:13.455270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:13.455971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.456250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.456521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.466659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:16:13.474312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:13.475426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:16:13.476482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 571/1149] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:16:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:13.481603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:13.579348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ecbd7-03d1-4390-acd2-5a35cea8560b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:13.612715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:13.613400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.613690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.613969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.614723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:13.615032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:13.616404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:13.621597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:13.626129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 579/1150] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:13.632037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:13.671064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:13.751480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aee6b-35de-41ba-950c-3b328a32305b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:13.794041 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:13.794899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.795176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.795444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] PATH_INFO: `/v3/users/bbfcd1ec15234f6e9226ea9ddebf96d0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.796679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] RBAC: Authorizing `identity:get_user(user_id=bbfcd1ec15234f6e9226ea9ddebf96d0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:13.798156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:13.799834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 572/1151] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:13 2022] GET /identity/v3/users/bbfcd1ec15234f6e9226ea9ddebf96d0 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:13.805842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:13.905607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96af8-41fb-49cb-baae-9669ff016219 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:13.939716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759247e-51fd-40c1-9c19-5eefd21460c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:13.940581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759247e-51fd-40c1-9c19-5eefd21460c5 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.940868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759247e-51fd-40c1-9c19-5eefd21460c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.941177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759247e-51fd-40c1-9c19-5eefd21460c5 None admin] PATH_INFO: `/v3/projects/29e6b2b8ed104be5a5cc461a63936813` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.942358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759247e-51fd-40c1-9c19-5eefd21460c5 None admin] RBAC: Authorizing `identity:get_project(project_id=29e6b2b8ed104be5a5cc461a63936813)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:13.943794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759247e-51fd-40c1-9c19-5eefd21460c5 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:13.945551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 580/1152] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:13 2022] GET /identity/v3/projects/29e6b2b8ed104be5a5cc461a63936813 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:13.951740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:14.053030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc0b09-c0ee-472e-a5f8-f3bc9d87ea0d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:14.054618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:14.088471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476fe4d-779c-4e22-864c-265d14d11517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:14.089388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476fe4d-779c-4e22-864c-265d14d11517 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:14.089673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476fe4d-779c-4e22-864c-265d14d11517 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:14.089946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476fe4d-779c-4e22-864c-265d14d11517 None admin] PATH_INFO: `/v3/projects/29e6b2b8ed104be5a5cc461a63936813/users/bbfcd1ec15234f6e9226ea9ddebf96d0/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:14.091889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476fe4d-779c-4e22-864c-265d14d11517 None admin] RBAC: Authorizing `identity:create_grant(project_id=29e6b2b8ed104be5a5cc461a63936813, user_id=bbfcd1ec15234f6e9226ea9ddebf96d0, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:14.093442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476fe4d-779c-4e22-864c-265d14d11517 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:14.106985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 573/1153] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:16:13 2022] PUT /identity/v3/projects/29e6b2b8ed104be5a5cc461a63936813/users/bbfcd1ec15234f6e9226ea9ddebf96d0/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:16:16.171781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47898d7-a747-49b6-a040-8334e90f80f1 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.172359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47898d7-a747-49b6-a040-8334e90f80f1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.172359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47898d7-a747-49b6-a040-8334e90f80f1 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.173749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 581/1154] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:16.179457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.179870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.180017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.196636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:16.197865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:16.263053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 574/1155] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:16:16.286971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47898d7-a747-49b6-a040-8334e90f80f1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:16.358210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:16.392733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47898d7-a747-49b6-a040-8334e90f80f1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:16.427022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:16.428098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.428612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.429152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.442317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:16:16.449979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:16.451085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:16:16.452650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 582/1156] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:16.457834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:16.552942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:16.563076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a7680-962b-454a-84b1-eee5fa34b64d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:16.597459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:16.598123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.598409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.598688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.599437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:16.599748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:16.601147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:16.609832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 575/1157] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:16.615583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:16.713822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e075e-2fd2-43a1-abdd-3766db953178 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:16.740797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:16.748816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:16.749719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.750000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.750277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] PATH_INFO: `/v3/users/bbfcd1ec15234f6e9226ea9ddebf96d0` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.751465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] RBAC: Authorizing `identity:get_user(user_id=bbfcd1ec15234f6e9226ea9ddebf96d0)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:16.752949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:16.754586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 583/1158] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:16 2022] GET /identity/v3/users/bbfcd1ec15234f6e9226ea9ddebf96d0 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:16.760487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:16.865881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbee466-a2c3-40ae-80a3-309d16465924 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:16.898216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:16.898860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.899151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.899429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.924478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:16.966576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:16.968623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:17.065303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 576/1159] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:16:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:16:17.071196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:17.177978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72185935-fccb-4486-b68d-ccd7528bd81e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:17.213799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639753e-8e29-4c2c-9f43-d796ec3c6870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:17.214652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639753e-8e29-4c2c-9f43-d796ec3c6870 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.214936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639753e-8e29-4c2c-9f43-d796ec3c6870 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.215212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639753e-8e29-4c2c-9f43-d796ec3c6870 None admin] PATH_INFO: `/v3/projects/29e6b2b8ed104be5a5cc461a63936813` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.216377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639753e-8e29-4c2c-9f43-d796ec3c6870 None admin] RBAC: Authorizing `identity:get_project(project_id=29e6b2b8ed104be5a5cc461a63936813)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:17.218164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639753e-8e29-4c2c-9f43-d796ec3c6870 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:17.219825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 584/1160] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:17 2022] GET /identity/v3/projects/29e6b2b8ed104be5a5cc461a63936813 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:17.226017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:17.281031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:17.325571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c906075-af26-458e-b5da-29927b261f1b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:17.359304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:17.359951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.360238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.360517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.361383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=29e6b2b8ed104be5a5cc461a63936813, user.id=bbfcd1ec15234f6e9226ea9ddebf96d0) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:17.361700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:17.363160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bba8c5-be37-4e7a-a8ff-3167bfab234b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:17.369048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 577/1161] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:16:17 2022] GET /identity/v3/role_assignments?user.id=bbfcd1ec15234f6e9226ea9ddebf96d0&scope.project.id=29e6b2b8ed104be5a5cc461a63936813&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:19.434027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906392-6a3f-468f-99f8-55cdc413ad37 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:19.434567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906392-6a3f-468f-99f8-55cdc413ad37 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:19.434567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2906392-6a3f-468f-99f8-55cdc413ad37 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.435979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 585/1162] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:19.441578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:19.442275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:19.442275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.459654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:19.460857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:19.529682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 578/1163] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:19.552650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906392-6a3f-468f-99f8-55cdc413ad37 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:19.619149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:19.658584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2906392-6a3f-468f-99f8-55cdc413ad37 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:19.694970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:19.695649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:19.696018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:19.696507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.769822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:19.771288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:19.832487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:19.871864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 586/1164] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:16:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:16:19.878828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:19.928052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:19.979973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf449b4-5f0a-42bf-864f-b78b23f10a7b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:20.015731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:20.016531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:20.016814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:20.017121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:20.024746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:20.026241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-66b516ce-c82a-493f-a7d9-8a59102292ba None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:20.041335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:20.042634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:16:20.043800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 579/1165] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:16:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:20.049550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:20.155411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84717e1c-9f21-4ed3-a7c0-2783c76e75cb None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:20.191265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:20.191919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:20.192216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:20.192480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:20.193228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:20.193536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:20.194896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018fbaa-3bbd-4456-9fb8-9b7ba85e4c2f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:20.202149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 587/1166] 10.222.0.79 () {56 vars in 1115 bytes} [Wed Jul 20 19:16:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:22.246926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d16f6c-71c0-4a39-8e37-34400316af0a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:22.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d16f6c-71c0-4a39-8e37-34400316af0a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:22.247504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d16f6c-71c0-4a39-8e37-34400316af0a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:22.249611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 580/1167] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:22.254997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be573be9-56e7-4b91-bf56-b9c84832325f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:22.255417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be573be9-56e7-4b91-bf56-b9c84832325f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:22.255560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be573be9-56e7-4b91-bf56-b9c84832325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:22.273691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-be573be9-56e7-4b91-bf56-b9c84832325f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:22.274875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-be573be9-56e7-4b91-bf56-b9c84832325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:22.298485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:22.341891 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 588/1168] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:22.364770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d16f6c-71c0-4a39-8e37-34400316af0a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:22.466204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d16f6c-71c0-4a39-8e37-34400316af0a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:22.474876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:22.501117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2cf29-72f2-4172-9d6c-813d8c2df58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:22.501704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2cf29-72f2-4172-9d6c-813d8c2df58b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:22.501989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2cf29-72f2-4172-9d6c-813d8c2df58b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:22.502265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2cf29-72f2-4172-9d6c-813d8c2df58b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:22.502997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2cf29-72f2-4172-9d6c-813d8c2df58b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:22.504361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2cf29-72f2-4172-9d6c-813d8c2df58b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:22.518514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 581/1169] 10.222.0.79 () {60 vars in 1138 bytes} [Wed Jul 20 19:16:22 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:16:24.615817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0f9c-3dcc-4733-abb9-18375b85c49f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.616466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0f9c-3dcc-4733-abb9-18375b85c49f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.616466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0f9c-3dcc-4733-abb9-18375b85c49f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.617777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 589/1170] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:24.622903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.623343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.623910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.643380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:24.644572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.713260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 582/1171] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:24.737387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c0f9c-3dcc-4733-abb9-18375b85c49f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:24.835433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c0f9c-3dcc-4733-abb9-18375b85c49f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:24.868117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b686a21-4840-4c71-bc5e-09bf73ae7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:24.868914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b686a21-4840-4c71-bc5e-09bf73ae7685 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.869206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b686a21-4840-4c71-bc5e-09bf73ae7685 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.869482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b686a21-4840-4c71-bc5e-09bf73ae7685 None admin] PATH_INFO: `/v3/domains/6333aa472c664e92bf6f905ad7427e93` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.879614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b686a21-4840-4c71-bc5e-09bf73ae7685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6333aa472c664e92bf6f905ad7427e93)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:24.881159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b686a21-4840-4c71-bc5e-09bf73ae7685 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:24.882807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 590/1172] 10.222.0.79 () {56 vars in 1183 bytes} [Wed Jul 20 19:16:24 2022] GET /identity/v3/domains/6333aa472c664e92bf6f905ad7427e93 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:24.888610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:24.982372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.987205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09565c-6bd1-49ad-8e57-0d8c09be2e8b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:25.021479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff203f-74d3-43bb-ab41-7ddd4d9e33a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:25.022127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff203f-74d3-43bb-ab41-7ddd4d9e33a7 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:25.022409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff203f-74d3-43bb-ab41-7ddd4d9e33a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:25.022681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff203f-74d3-43bb-ab41-7ddd4d9e33a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:25.023372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff203f-74d3-43bb-ab41-7ddd4d9e33a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:25.024783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff203f-74d3-43bb-ab41-7ddd4d9e33a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:25.051796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 583/1173] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:16:24 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:16:27.109774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c248-5ad9-4f73-bd5b-29a3b7e1bc10 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.110335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c248-5ad9-4f73-bd5b-29a3b7e1bc10 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.110335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c248-5ad9-4f73-bd5b-29a3b7e1bc10 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.111851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 591/1174] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:27.117383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.117854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.117967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.134889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:27.136087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:27.205940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 584/1175] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:27.231743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c248-5ad9-4f73-bd5b-29a3b7e1bc10 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:27.252296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:27.335773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c248-5ad9-4f73-bd5b-29a3b7e1bc10 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:27.370515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a2e5-3421-4f6c-9e02-7a4a82b49bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:27.371299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a2e5-3421-4f6c-9e02-7a4a82b49bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.371578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a2e5-3421-4f6c-9e02-7a4a82b49bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.371846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a2e5-3421-4f6c-9e02-7a4a82b49bb8 None admin] PATH_INFO: `/v3/domains/6333aa472c664e92bf6f905ad7427e93` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.373151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a2e5-3421-4f6c-9e02-7a4a82b49bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6333aa472c664e92bf6f905ad7427e93)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:27.375154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a2e5-3421-4f6c-9e02-7a4a82b49bb8 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:27.376795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 592/1176] 10.222.0.79 () {56 vars in 1183 bytes} [Wed Jul 20 19:16:27 2022] GET /identity/v3/domains/6333aa472c664e92bf6f905ad7427e93 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:27.382592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:27.462260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:27.482622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dedbf0-37fa-4029-967c-0308173ec45f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:27.516065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43127fa-9c59-4f53-b6ec-d115e734e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:27.516723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43127fa-9c59-4f53-b6ec-d115e734e17b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:27.517039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43127fa-9c59-4f53-b6ec-d115e734e17b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:27.517316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43127fa-9c59-4f53-b6ec-d115e734e17b None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:27.518024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43127fa-9c59-4f53-b6ec-d115e734e17b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:27.519460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43127fa-9c59-4f53-b6ec-d115e734e17b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:27.539930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 585/1177] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:16:27 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:16:29.677344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80334-1384-4a6e-a82c-780b0f7d5437 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:29.677914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80334-1384-4a6e-a82c-780b0f7d5437 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:29.677914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80334-1384-4a6e-a82c-780b0f7d5437 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:29.679281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 593/1178] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:29.684993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:29.685463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:29.685558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:29.703717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:29.704969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:29.787589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 586/1179] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:29.812903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80334-1384-4a6e-a82c-780b0f7d5437 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:29.907129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:29.925669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80334-1384-4a6e-a82c-780b0f7d5437 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:29.963133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:29.964009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:29.964296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:29.964574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:29.978594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:16:29.987820 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:29.988952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:16:29.989946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 594/1180] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:16:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:29.995404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:30.064560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.112222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ab466-b7b5-4b70-89bf-78c243589908 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:30.146202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:30.146869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.147154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.147432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.148180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:30.148491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:30.149895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:30.158601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 587/1181] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:30.164425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:30.264052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbc7fc-ecb8-4db3-ba77-791c2595ce6a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:30.298093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:30.298960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.299244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.299518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] PATH_INFO: `/v3/users/e0bc355fb6c34d85811f9e0c30a7e6d9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.312343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=e0bc355fb6c34d85811f9e0c30a7e6d9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:30.313809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:30.315483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 595/1182] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:30 2022] GET /identity/v3/users/e0bc355fb6c34d85811f9e0c30a7e6d9 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:30.321277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:30.422034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a10f5-8e9e-4139-89cd-2b5fcde14680 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:30.446988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.454947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:30.455587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.455866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.456144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.521863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:30.523306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:30.620004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 588/1183] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:16:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:16:30.625997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:30.629734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.732850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d420-dd7c-45c6-b672-580198bf6ac6 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:30.766797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05d84e-0b15-4ff9-a6ae-bd0b7e2cbe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:30.767648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d84e-0b15-4ff9-a6ae-bd0b7e2cbe45 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.767926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d84e-0b15-4ff9-a6ae-bd0b7e2cbe45 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.768195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d84e-0b15-4ff9-a6ae-bd0b7e2cbe45 None admin] PATH_INFO: `/v3/projects/60dac0f5b77144c5a92e82150e22d92e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.769378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05d84e-0b15-4ff9-a6ae-bd0b7e2cbe45 None admin] RBAC: Authorizing `identity:get_project(project_id=60dac0f5b77144c5a92e82150e22d92e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:30.770794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05d84e-0b15-4ff9-a6ae-bd0b7e2cbe45 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:30.772536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 596/1184] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:30 2022] GET /identity/v3/projects/60dac0f5b77144c5a92e82150e22d92e => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:30.778809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:30.806362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:30.882019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146121d8-1afb-441c-ac52-471428e4e527 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:30.915910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:30.916553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:30.916834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:30.917139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:30.917959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=60dac0f5b77144c5a92e82150e22d92e, user.id=e0bc355fb6c34d85811f9e0c30a7e6d9) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:30.918272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:30.919692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8110f-9128-44fb-bcf5-ea9bf8faf732 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:30.925209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 589/1185] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:16:30 2022] GET /identity/v3/role_assignments?user.id=e0bc355fb6c34d85811f9e0c30a7e6d9&scope.project.id=60dac0f5b77144c5a92e82150e22d92e&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:32.978235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404dc56-ef3c-4570-9987-b5e137865974 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:32.978758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404dc56-ef3c-4570-9987-b5e137865974 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:32.978758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404dc56-ef3c-4570-9987-b5e137865974 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:32.980080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 597/1186] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:32.985737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:32.986167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:32.986305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.003208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:33.004394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:33.069956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 590/1187] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:33.093586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404dc56-ef3c-4570-9987-b5e137865974 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:33.165914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:33.197178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404dc56-ef3c-4570-9987-b5e137865974 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:33.230093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:33.230815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:33.231100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:33.231375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.242072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:16:33.249978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:33.251090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:16:33.252142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 598/1188] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:33.257344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:33.282531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:33.358241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d20388-601c-44d7-a999-b6a896c34b12 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:33.393258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:33.393926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:33.394213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:33.394490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.395273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:33.395584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:33.397171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:33.405549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 591/1189] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:33.411398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:33.510254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738a812-87b1-4c0c-9919-1f86c731aa55 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:33.539325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:33.544856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:33.546325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:33.546611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:33.546887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] PATH_INFO: `/v3/users/e0bc355fb6c34d85811f9e0c30a7e6d9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.548090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] RBAC: Authorizing `identity:get_user(user_id=e0bc355fb6c34d85811f9e0c30a7e6d9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:33.549567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:33.551221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 599/1190] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:33 2022] GET /identity/v3/users/e0bc355fb6c34d85811f9e0c30a7e6d9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:33.557214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:33.639389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:33.655652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd1004-f3c5-44cb-a6c1-d3f7cfdc1367 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:33.689301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a8604-e2f4-4c67-951e-7afdb55f8291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:33.690164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a8604-e2f4-4c67-951e-7afdb55f8291 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:33.690446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a8604-e2f4-4c67-951e-7afdb55f8291 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:33.690719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a8604-e2f4-4c67-951e-7afdb55f8291 None admin] PATH_INFO: `/v3/projects/60dac0f5b77144c5a92e82150e22d92e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.691885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a8604-e2f4-4c67-951e-7afdb55f8291 None admin] RBAC: Authorizing `identity:get_project(project_id=60dac0f5b77144c5a92e82150e22d92e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:33.693334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a8604-e2f4-4c67-951e-7afdb55f8291 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:33.695132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 592/1191] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:33 2022] GET /identity/v3/projects/60dac0f5b77144c5a92e82150e22d92e => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:33.701164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:33.798807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7270e4-c009-4286-a314-5a6b35f8abae None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:33.831841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb3ed0-cc26-48cc-a6bc-87c8c9199a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:33.832718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb3ed0-cc26-48cc-a6bc-87c8c9199a9b None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:33.833039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb3ed0-cc26-48cc-a6bc-87c8c9199a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:33.833320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb3ed0-cc26-48cc-a6bc-87c8c9199a9b None admin] PATH_INFO: `/v3/projects/60dac0f5b77144c5a92e82150e22d92e/users/e0bc355fb6c34d85811f9e0c30a7e6d9/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:33.835270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb3ed0-cc26-48cc-a6bc-87c8c9199a9b None admin] RBAC: Authorizing `identity:create_grant(project_id=60dac0f5b77144c5a92e82150e22d92e, user_id=e0bc355fb6c34d85811f9e0c30a7e6d9, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:33.836711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb3ed0-cc26-48cc-a6bc-87c8c9199a9b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:33.850390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 600/1192] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:16:33 2022] PUT /identity/v3/projects/60dac0f5b77144c5a92e82150e22d92e/users/e0bc355fb6c34d85811f9e0c30a7e6d9/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:16:35.861721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715f3ef-6cc6-48ab-b4a6-032ed57ec26e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:35.862271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715f3ef-6cc6-48ab-b4a6-032ed57ec26e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:35.862271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715f3ef-6cc6-48ab-b4a6-032ed57ec26e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:35.863641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 593/1193] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:35.868847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:35.869275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:35.869441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:35.886027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:35.887220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:35.942416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:35.954727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 601/1194] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:16:35.978144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715f3ef-6cc6-48ab-b4a6-032ed57ec26e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:36.084180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715f3ef-6cc6-48ab-b4a6-032ed57ec26e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:36.117156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:36.117937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:36.118219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:36.118520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:36.129749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:16:36.137302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:16:36.138416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:16:36.139517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 594/1195] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:16:36.144838 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:36.244759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefbf3d-858c-4e30-9258-1888c28cb7b9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:36.278032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:36.278702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:36.279119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:36.280387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:36.281722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:36.282228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:36.284683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:36.294316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 602/1196] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:36.301448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:36.317177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:36.401828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc165f-08de-4e40-9b18-d7d9d61b3467 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:36.442161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:36.443297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:36.443583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:36.443855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] PATH_INFO: `/v3/users/e0bc355fb6c34d85811f9e0c30a7e6d9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:36.445046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] RBAC: Authorizing `identity:get_user(user_id=e0bc355fb6c34d85811f9e0c30a7e6d9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:36.446474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:36.448098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 595/1197] 10.222.0.79 () {56 vars in 1177 bytes} [Wed Jul 20 19:16:36 2022] GET /identity/v3/users/e0bc355fb6c34d85811f9e0c30a7e6d9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:36.453937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:36.510952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:36.566955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b5a9-1c2f-4630-ba4b-52a09da895ba None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:36.602005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:36.602657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:36.602943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:36.603215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:36.673018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:36.674467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:36.729792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:36.777697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 603/1198] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:16:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:16:36.784738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:36.865934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:36.885537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923c82-7804-4ef4-82d4-ecd388d17d2a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:36.920552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6ce06-113a-4393-b559-48d7039ba0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:36.921466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6ce06-113a-4393-b559-48d7039ba0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:36.921750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6ce06-113a-4393-b559-48d7039ba0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:36.922025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6ce06-113a-4393-b559-48d7039ba0e8 None admin] PATH_INFO: `/v3/projects/60dac0f5b77144c5a92e82150e22d92e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:36.923194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6ce06-113a-4393-b559-48d7039ba0e8 None admin] RBAC: Authorizing `identity:get_project(project_id=60dac0f5b77144c5a92e82150e22d92e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:36.924611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6ce06-113a-4393-b559-48d7039ba0e8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:36.926331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 596/1199] 10.222.0.79 () {56 vars in 1186 bytes} [Wed Jul 20 19:16:36 2022] GET /identity/v3/projects/60dac0f5b77144c5a92e82150e22d92e => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:36.932500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:37.030641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05c4f-8389-4618-badd-289868730b83 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:37.064554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:37.065232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:37.065519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:37.065797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:37.066624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=60dac0f5b77144c5a92e82150e22d92e, user.id=e0bc355fb6c34d85811f9e0c30a7e6d9) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:16:37.066938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:37.068371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36af947-b3b8-432d-b6c1-2464c5de02df None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:37.074614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 604/1200] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:16:36 2022] GET /identity/v3/role_assignments?user.id=e0bc355fb6c34d85811f9e0c30a7e6d9&scope.project.id=60dac0f5b77144c5a92e82150e22d92e&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:16:46.916007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=26 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:47.071518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:33.402174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790eb8-48f6-4ccd-bc27-c3173c38bcb1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:33.402174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790eb8-48f6-4ccd-bc27-c3173c38bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:33.402174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790eb8-48f6-4ccd-bc27-c3173c38bcb1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:33.403360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 597/1201] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:33.409367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:33.410253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:33.410529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:33.430838 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:33.432128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:33.542924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 605/1202] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:17:33.575546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790eb8-48f6-4ccd-bc27-c3173c38bcb1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:33.731812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790eb8-48f6-4ccd-bc27-c3173c38bcb1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:33.778610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:33.779996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:33.780236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:33.780517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:33.793607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:33.796494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:33.818706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:33.820047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:33.821120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:33.821120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:33.821120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:33.821120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:33.821120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 598/1203] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:33.827147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:33.963437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f652455-f816-4c02-86f9-78cffc26a344 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:34.004947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:34.018120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:34.018935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:34.019264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:34.019543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:34.020419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:34.020902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:34.022963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7991a8d-910c-42cb-8b83-d5bd62b48b20 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:34.042537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 606/1204] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:34.050285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:34.125768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:34.188832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014395c-5828-4823-aa80-fed716d54354 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:34.236575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cf9e2-459f-4a6c-bfe0-9797a9c3db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:34.237329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cf9e2-459f-4a6c-bfe0-9797a9c3db6f None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:34.237674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cf9e2-459f-4a6c-bfe0-9797a9c3db6f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:34.238017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cf9e2-459f-4a6c-bfe0-9797a9c3db6f None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:34.238872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cf9e2-459f-4a6c-bfe0-9797a9c3db6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:34.240435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cf9e2-459f-4a6c-bfe0-9797a9c3db6f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:34.272063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 599/1205] 10.222.0.79 () {60 vars in 1132 bytes} [Wed Jul 20 19:17:34 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:17:36.683093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb399e-c412-455f-b764-0a759f8031a6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:36.683690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb399e-c412-455f-b764-0a759f8031a6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:36.683690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb399e-c412-455f-b764-0a759f8031a6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:36.685787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 607/1206] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:36.691774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:36.692269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:36.692365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:36.711952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:36.713252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:36.803503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 600/1207] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:17:36.830739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb399e-c412-455f-b764-0a759f8031a6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:36.959344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb399e-c412-455f-b764-0a759f8031a6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:36.998113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:36.999068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:36.999350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:36.999661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:37.012034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:17:37.020494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:37.021614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:37.023367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 608/1208] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:17:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:37.028507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:37.071949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:37.144090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376a28-cd0f-4f36-8819-290aa41a1d86 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:37.189211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:37.189942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:37.190252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:37.190528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:37.191287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:37.191594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:37.193195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:37.203497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 601/1209] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:17:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:37.209307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:37.268039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:37.346968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb2402-d18e-44bd-8829-a74205b864fd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:37.393068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:37.394170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:37.394577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:37.394941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:37.412960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:37.415022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:37.441872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:37.442919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:37.444199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:37.444199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:37.444199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:37.444199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:37.448141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 609/1210] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:17:37.449218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:37.566692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:37.571194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba3785-ca1d-4d65-90f6-df99f1e37ad8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:37.633988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:37.634808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:37.635016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:37.635295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:37.636047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:37.636345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:37.637882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:37.651639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 602/1211] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:37.658518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:37.792461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4ecc2-8074-4cce-a455-598c9b974cdc None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:37.794238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:37.834143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:37.835078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:37.835502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:37.835887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:37.845040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:37.846700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:17:37.865713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:37.867042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:37.868209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 610/1212] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:37.873410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:37.998775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e708f5-6790-4fe2-98c4-757ab5eae6a7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:38.040261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:38.042609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:38.042609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:38.042609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:38.042609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:38.042952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:38.044473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:38.072105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 603/1213] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Jul 20 19:17:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:38.073413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:38.198625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c565f-6df0-4ae3-ad2a-187d8458a30a None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:38.240047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:38.240047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:38.240425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:38.240522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:38.276413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:38.322396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:38.324004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:38.441030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 611/1214] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:17:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:17:38.446716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:38.467783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:38.564096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a522f5-3dba-425d-b9ff-77ef7121e984 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:38.608672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:38.609734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:38.609842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:38.610160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:38.623188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:38.625333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:38.648732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:38.650010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:38.651114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:38.651114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:38.651114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:38.651114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:38.652324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 604/1215] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:17:38.659514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:38.729056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:38.775197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09a3a-1d9b-4c16-85ba-3c74f3c4726f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:38.817304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:38.817987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:38.818296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:38.818519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:38.819388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:38.819697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:38.821232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:38.834993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 612/1216] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:17:38.841422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:38.926134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:38.967577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a395-5470-486e-b823-eb3680c2d12b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:39.012133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:39.013011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:39.013853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:39.014233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:39.022219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:39.023763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:17:39.040208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:39.041453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:39.042373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 605/1217] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:39.047716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:39.171913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cbf87-7d82-4c61-a525-65ec86257d7b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:39.202094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:39.215517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:39.216213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:39.216503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:39.217036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:39.217837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:39.218155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:39.219582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae81777-1284-4539-bdee-6249250f9e09 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:39.237482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 613/1218] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:39.239679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:39.360485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839daa4c-04fe-41b7-b2d1-441692e7b390 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:39.397396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:39.409949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:39.410706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:39.411015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:39.411363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:39.412607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=add72a4c345b4ab49e822f478eebd6b8) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:39.412789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:39.414525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959c5a1-13f1-42b6-8911-410afb392d5c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:39.432909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 606/1219] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:17:39 2022] GET /identity/v3/role_assignments?user.id=add72a4c345b4ab49e822f478eebd6b8&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:17:41.456259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612ad3f-10d2-4ee7-86ee-ecb9acd02838 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.457077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612ad3f-10d2-4ee7-86ee-ecb9acd02838 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.457077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612ad3f-10d2-4ee7-86ee-ecb9acd02838 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.458208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 614/1220] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:41.463442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.463923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.464046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.482724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:41.484115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:41.566745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 607/1221] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:17:41.593158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612ad3f-10d2-4ee7-86ee-ecb9acd02838 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:41.719612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612ad3f-10d2-4ee7-86ee-ecb9acd02838 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:41.758574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:41.759463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.759781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.760120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.771964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:17:41.780668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:41.781710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:41.782577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 615/1222] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:17:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:41.788403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:41.895653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653fcc3-6672-4a72-84f1-815adc43405b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:41.928462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:41.939884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:41.940526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:41.940809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:41.941130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:41.941907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:41.942215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:41.943593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:41.952497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 608/1223] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:17:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:41.958848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:42.004117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:42.068485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d80cdf-240a-4192-ab10-1d81981bc0d3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:42.102812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:42.103608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:42.103892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:42.104168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:42.115449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:42.116980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:42.134449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:42.135419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:42.136338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:42.136338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:42.136338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:42.136338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:42.136338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 616/1224] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:42.143529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:42.250025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465aeab-003e-496d-a03b-6fda7a830be6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:42.287796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:42.288483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:42.288749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:42.289144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:42.289877 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:42.290186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:42.291550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:42.304466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 609/1225] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:42.310948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:42.430919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:42.439079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0be4a-d4eb-46a1-889e-105b58efacd0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:42.479452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:42.480299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:42.480578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:42.480857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:42.489925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:42.491529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:17:42.506969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:42.508238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:42.509264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 617/1226] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:42.515305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:42.566490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:42.628437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82039b4f-1544-4c0a-9da3-9b0058f4a4ce None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:42.667053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:42.667748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:42.668030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:42.668300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:42.669204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:42.670162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:42.671590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:42.691016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 610/1227] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Jul 20 19:17:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:42.696772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:42.802417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f466b3-54e3-4e6f-a5bd-d234650b74aa None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:42.840464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:42.841289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:42.841577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:42.841858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:42.852616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:42.854114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:42.871706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:42.873193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:42.874099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:42.874099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:42.874099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:42.874099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:42.874099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 618/1228] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:42.879691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:42.995249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7433c6-6176-4b0f-99aa-78fdd634e08f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:42.998164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:43.034803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:43.035495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:43.035757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:43.036034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:43.036764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:43.037108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:43.038510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:43.051377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 611/1229] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:43.057455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:43.068121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:43.167989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b950766-fcf3-4fb6-9774-0bd5ecaaae27 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:43.208044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:43.208865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:43.209178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:43.209476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:43.218944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:43.220496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:17:43.237532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:43.238629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:43.239695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:43.244909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 619/1230] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:43.245874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:43.358830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b532fe2-a9ce-49c1-8734-71a6096b787f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:43.399410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:43.400342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:43.400638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:43.401030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:43.401982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:43.402294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:43.404406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1c7a9-d7bf-4a49-80f6-12e296722454 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:43.419549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 612/1231] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:43.425749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:43.503590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:43.545461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314c48a-905d-4673-90cf-38cdfe6af905 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:43.582776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214e137-df0e-4ec8-84e3-817ed1b47836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:43.583677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214e137-df0e-4ec8-84e3-817ed1b47836 None admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:43.584020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214e137-df0e-4ec8-84e3-817ed1b47836 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:43.584311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214e137-df0e-4ec8-84e3-817ed1b47836 None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/add72a4c345b4ab49e822f478eebd6b8/roles/b46d941de5b94259ae2656880e02f511` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:43.600074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214e137-df0e-4ec8-84e3-817ed1b47836 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=add72a4c345b4ab49e822f478eebd6b8, role_id=b46d941de5b94259ae2656880e02f511)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:43.601564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214e137-df0e-4ec8-84e3-817ed1b47836 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:43.614170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 620/1232] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:17:43 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/add72a4c345b4ab49e822f478eebd6b8/roles/b46d941de5b94259ae2656880e02f511 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:17:45.844388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7f59-df07-453f-a920-2f436e116e26 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.844962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7f59-df07-453f-a920-2f436e116e26 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.844962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7f59-df07-453f-a920-2f436e116e26 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.846374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 613/1233] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:45.851828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.852339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.852434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.869374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:45.870599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:45.940634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 621/1234] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:17:45.964651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7f59-df07-453f-a920-2f436e116e26 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:45.992903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.075980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7f59-df07-453f-a920-2f436e116e26 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:46.118564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:46.119512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.119855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.120237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.132175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:17:46.140690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:46.141867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:46.142816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 614/1235] 10.222.0.79 () {56 vars in 1102 bytes} [Wed Jul 20 19:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:46.148361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:46.198885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.257827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeef8f-b04d-404a-8eba-2387b3e95267 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:46.293403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:46.294085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.294376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.294681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.295505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:46.295812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:46.297217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:46.305952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 622/1236] 10.222.0.79 () {56 vars in 1103 bytes} [Wed Jul 20 19:17:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:46.312597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:46.406754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.426856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492b63-85d9-40cd-9b69-56e4732e949a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:46.472312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:46.473142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.473426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.473700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.486286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:46.488254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:46.509138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:46.510218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:46.511307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:46.511307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:46.511307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:46.511307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:46.511307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 615/1237] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:46.517247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:46.623806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9ed38-b95c-49ce-b565-2113025c55c9 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:46.633507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.661451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:46.662150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.662412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.662680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.663397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:46.663703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:46.665099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:46.677095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 623/1238] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:46.683081 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:46.803817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc175b-c707-41c2-8650-7fa23101c233 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:46.845963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:46.846792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:46.847102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:46.847380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:46.850385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:46.856085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:46.858394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:17:46.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:46.875177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:46.876646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 616/1239] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:46.881449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:46.996338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da031023-a42c-4e26-84c7-0c9110902920 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:47.033069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:47.033749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:47.034037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:47.034312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:47.035187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:47.035477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:47.036938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:47.062717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 624/1240] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Jul 20 19:17:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:47.064059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:47.182703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd40568-821c-4998-83f9-0a1ca79aa2a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:47.222858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:47.223502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:47.223784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:47.224057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:47.299079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:47.306969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:47.308629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:47.427802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 617/1241] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:17:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:17:47.434020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:47.464135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:47.542343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8484d6-25c4-432a-912c-93cfad0a9ad3 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:47.579177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:47.580010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:47.580298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:47.580577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:47.592351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:47.593854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:47.612052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:47.613200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:47.614132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:47.614132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:47.614132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:47.614132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:47.614132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 625/1242] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:47.619131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:47.738872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:47.749454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08b5aa-ac1c-45a3-a24a-a5a02728c8f7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:47.792177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:47.792993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:47.793277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:47.793553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:47.794590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:47.794978 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:47.796591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:47.811124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 618/1243] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:47.816946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:47.904956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:47.928785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c5c12-eeec-4b46-8de6-d502ada71190 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:47.972688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:47.973534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:47.973893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:47.974197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:47.982540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:47.984277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:17:47.999734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:48.000934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:48.002211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 626/1244] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:48.006854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:48.119244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72bd2-906b-433e-98ff-1f3fab2867e0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:48.158788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:48.159592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:48.160029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:48.160349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:48.161280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:48.161599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:48.163029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c67e1-aa55-45b4-ad28-9422d21ef48b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:48.175900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 619/1245] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:17:48.182855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:48.302682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651d2e5-de67-4280-b6ea-ed0ebab5129d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:48.348452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:48.349356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:48.349669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:48.350015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:48.351024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] RBAC: Adding query filter params (role.id=b46d941de5b94259ae2656880e02f511, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=add72a4c345b4ab49e822f478eebd6b8) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:48.351441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:48.353160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1406b04-a558-40a5-8914-11c355d05979 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:48.362108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 627/1246] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:17:48 2022] GET /identity/v3/role_assignments?user.id=add72a4c345b4ab49e822f478eebd6b8&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=b46d941de5b94259ae2656880e02f511 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:50.828704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b705-2159-4f84-8b27-649fb7d7cc0d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:50.829292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b705-2159-4f84-8b27-649fb7d7cc0d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:50.829402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b705-2159-4f84-8b27-649fb7d7cc0d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:50.830986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 620/1247] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:50.837291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:50.837769 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:50.837971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:50.844082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:50.857498 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:50.859086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:50.942898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 628/1248] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:17:50.970929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b705-2159-4f84-8b27-649fb7d7cc0d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:50.990590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:51.088016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b705-2159-4f84-8b27-649fb7d7cc0d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:51.128917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb507-f769-488e-b378-a7645058493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:51.129723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb507-f769-488e-b378-a7645058493c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:51.130004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb507-f769-488e-b378-a7645058493c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:51.130276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb507-f769-488e-b378-a7645058493c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:51.142162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bb507-f769-488e-b378-a7645058493c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-000bb507-f769-488e-b378-a7645058493c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:17:51.149871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:51.151082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:51.151936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 621/1249] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:17:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:51.157765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:51.269991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03667062-cfa5-4233-bcdb-df4371c68ab4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:51.306723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:51.313107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:51.313784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:51.314070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:51.314344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:51.315095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:51.315403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:51.316767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:51.325491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 629/1250] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:17:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:17:51.331421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb507-f769-488e-b378-a7645058493c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:51.433278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:51.465831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb507-f769-488e-b378-a7645058493c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:51.503806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:51.504605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:51.504902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:51.505264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:51.517639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:51.519829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:51.538377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:51.539505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:51.540415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:51.540415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:51.540415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:51.540415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:51.540415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 622/1251] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:51.547478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:51.661038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17c5d7-e290-40da-a5b0-e236e84c2118 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:51.708976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:51.709649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:51.709936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:51.710228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:51.710954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:51.711265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:51.712676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:51.728109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 630/1252] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:51.735928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:51.849700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89091f21-2f73-4606-8db1-3f8d988f6d02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:51.877723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:51.886679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:51.887486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:51.887815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:51.888179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:51.896019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:51.897592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:17:51.914660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:51.915850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:51.921629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 623/1253] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:51.922233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:51.985164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:52.037901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301ba4d-b890-49eb-b343-b2f865e07804 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:52.077243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:52.077920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:52.078227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:52.078505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:52.079344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:52.079689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:52.081230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:52.106030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 631/1254] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Jul 20 19:17:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:52.107192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:52.226552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408d96b-5404-4e9a-8637-35daa14d3c19 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:52.269921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:52.270571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:52.270938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:52.271346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:52.336653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:52.351532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:52.353738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:52.470832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 624/1255] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:17:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:17:52.476709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:52.590374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f5df-ebd4-449d-9323-8d363386362b None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:52.603717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:52.628100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:52.628946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:52.629237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:52.629519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:52.640868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:52.642379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:52.660813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:52.661889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:52.663154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:52.663154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:52.663154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:52.663154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:52.663154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 632/1256] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:52.667822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:52.767895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:52.781410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97b250-9987-4386-b430-a14d921ca60e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:52.818043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:52.818764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:52.819030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:52.819310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:52.820033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:52.820339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:52.821739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:52.833613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 625/1257] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:52.839398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:52.947566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdce588-eb30-4eea-8cbb-081b9a7670cf None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:52.984568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:52.985402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:52.985691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:52.986006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:52.993985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:52.995527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:17:53.010982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:53.012099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:53.013580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 633/1258] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:53.018762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:53.131024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c9e79-044f-4d92-b06b-6761f3bbf0a2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:53.174528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:53.175339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:53.175628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:53.175905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:53.176812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:53.177151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:53.178587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8a367-f92f-405a-bdf8-56ab1a25c13e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:53.192993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 626/1259] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:17:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:17:53.200658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:53.228169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:53.311931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367126-4504-4597-bc48-5fcc30f5c582 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:53.349082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:53.349744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:53.350029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:53.350307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:53.351151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=add72a4c345b4ab49e822f478eebd6b8) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:53.351463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:53.352919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be62a5-ee64-43a9-aaab-6d5a29db396e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:53.359545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 634/1260] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:17:53 2022] GET /identity/v3/role_assignments?user.id=add72a4c345b4ab49e822f478eebd6b8&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=689534f62373409f9d02ea4acc335ab7 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:17:55.434265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca34cf3-4793-4475-8155-14a64894afac None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:55.434844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca34cf3-4793-4475-8155-14a64894afac None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:55.435032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca34cf3-4793-4475-8155-14a64894afac None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:55.436442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 627/1261] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:17:55.442436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d216100-c342-43db-85ae-014eb8488489 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:55.442871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d216100-c342-43db-85ae-014eb8488489 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:55.442995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d216100-c342-43db-85ae-014eb8488489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:55.459465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3d216100-c342-43db-85ae-014eb8488489 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:55.460653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3d216100-c342-43db-85ae-014eb8488489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:55.526003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 635/1262] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:17:55.550395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca34cf3-4793-4475-8155-14a64894afac None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:55.561675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:55.650954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca34cf3-4793-4475-8155-14a64894afac None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:55.686280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:55.687004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:55.687287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:55.687561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:55.698047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:17:55.706626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:55.707717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:55.708597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 628/1263] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:17:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:55.713725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d216100-c342-43db-85ae-014eb8488489 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:55.732975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:55.815887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d216100-c342-43db-85ae-014eb8488489 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:55.849963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:55.850624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:55.850906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:55.851180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:55.851919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:55.852229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:55.853609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:55.861893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 636/1264] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:55.867919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:55.938192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:55.974180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17adec3-0c68-4af2-a83c-1f4f165271d8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.011264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.012047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.012326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.013141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.024061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.025519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:56.042639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:56.043701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:56.045182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:56.045182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:56.045182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:56.045182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:56.045182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 629/1265] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:56.049993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.128544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:56.147758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b23c-6fb2-4388-804f-7ed96d1ed40d None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.180485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.181163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.181446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.181718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.182431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:56.182735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.184086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.196098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 637/1266] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:56.202566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.300583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a5883-02d7-4eb0-8fe1-86a0b097bd47 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.319959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:56.334635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.335432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.335720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.336956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.346016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.347467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:17:56.362385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:56.363522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:56.364698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 630/1267] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:56.369680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.487721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543860d-a644-4a79-b82c-14c2ced8b352 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.521107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.521768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.522052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.522327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.523158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:56.523462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.524904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.533956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:56.542268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 638/1268] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:56.548972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.647106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437861cc-70a6-490f-ba5d-3b9e165b706d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.679958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.680737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.681053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.681331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.692395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.693877 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:56.712618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:56.713818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:56.714918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:56.714918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:56.714918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:56.714918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:56.714918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 631/1269] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:56.719753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.819647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0683f-930d-4b6c-8045-9563435074fe None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.852791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535625a-3843-4809-9394-b614408e040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.853485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535625a-3843-4809-9394-b614408e040f None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.853768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535625a-3843-4809-9394-b614408e040f None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.854044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535625a-3843-4809-9394-b614408e040f None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.854766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535625a-3843-4809-9394-b614408e040f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:56.855172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535625a-3843-4809-9394-b614408e040f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.857175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535625a-3843-4809-9394-b614408e040f None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.868234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 639/1270] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:56.873969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.905620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:56.979018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c321f0-bfa8-4a67-8802-50d7e7a959d5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:57.012397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:57.013324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:57.013611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:57.013888 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:57.020605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:57.022081 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:17:57.037065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:57.038105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:17:57.039012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 632/1271] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:17:57.044743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535625a-3843-4809-9394-b614408e040f None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:57.101906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:57.143906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535625a-3843-4809-9394-b614408e040f None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:57.177905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:57.178557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:57.178839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:57.179111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:57.179888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:57.180192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:57.181630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640713-c72b-4175-99e0-a0594c8af604 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:57.193503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 640/1272] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:57.199490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:57.288956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:57.298093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ced6c-f0d0-4542-8e9e-bd32b287e62c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:57.331518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604902ae-b287-420c-9368-5f87e37089dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:57.332396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604902ae-b287-420c-9368-5f87e37089dd None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:57.333098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604902ae-b287-420c-9368-5f87e37089dd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:57.333098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604902ae-b287-420c-9368-5f87e37089dd None admin] PATH_INFO: `/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/add72a4c345b4ab49e822f478eebd6b8/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:57.334911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604902ae-b287-420c-9368-5f87e37089dd None admin] RBAC: Authorizing `identity:create_grant(project_id=ec730e6d5610427eb8e3d2ae8808c123, user_id=add72a4c345b4ab49e822f478eebd6b8, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:57.336332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604902ae-b287-420c-9368-5f87e37089dd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:57.349349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 633/1273] 10.222.0.79 () {58 vars in 1439 bytes} [Wed Jul 20 19:17:57 2022] PUT /identity/v3/projects/ec730e6d5610427eb8e3d2ae8808c123/users/add72a4c345b4ab49e822f478eebd6b8/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:17:59.384377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a17be-776a-4f5a-bc4f-ccc45a3594f1 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:59.384940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a17be-776a-4f5a-bc4f-ccc45a3594f1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:59.384940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a17be-776a-4f5a-bc4f-ccc45a3594f1 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:59.386358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 641/1274] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:59.392640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5f423-d756-4790-a725-599be75b031b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:59.393295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5f423-d756-4790-a725-599be75b031b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:59.393295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5f423-d756-4790-a725-599be75b031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:59.411585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cdd5f423-d756-4790-a725-599be75b031b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:59.412764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cdd5f423-d756-4790-a725-599be75b031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:59.485881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 634/1275] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:17:59.509951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a17be-776a-4f5a-bc4f-ccc45a3594f1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:59.613261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a17be-776a-4f5a-bc4f-ccc45a3594f1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:59.631421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:59.648450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:59.649206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:59.649493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:59.649768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:59.660154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:17:59.667837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:59.669465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:59.670452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 642/1276] 10.222.0.79 () {56 vars in 1096 bytes} [Wed Jul 20 19:17:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:17:59.675723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5f423-d756-4790-a725-599be75b031b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:59.757928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:59.776947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5f423-d756-4790-a725-599be75b031b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:59.811134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:59.811783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:59.812066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:59.812339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:59.813107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:17:59.813414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:59.814774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:59.822683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 635/1277] 10.222.0.79 () {56 vars in 1099 bytes} [Wed Jul 20 19:17:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:17:59.828950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:59.929743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1058d2-1c9d-4881-9481-9a04621fb3fd None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:59.962152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:59.962934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:59.963224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:59.963504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:59.974970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:59.976411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:17:59.993895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:17:59.994847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:17:59.995893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:17:59.995893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:17:59.995893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:17:59.995893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:17:59.995893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 643/1278] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:18:00.000912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:00.100353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a5506-0f52-42c5-b7dc-f6956f5e84ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:00.134475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:00.135132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:00.135420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:00.135700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:00.136422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:00.136732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:00.138220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:00.149751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 636/1279] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:00.156699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:00.187497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:00.256602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324840-a2e6-440c-9b25-d06927543f1e None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:00.290089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:00.290885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:00.291162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:00.291433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:00.298981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:00.300410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:18:00.314632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:18:00.315784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:00.317225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 644/1280] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:18:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:18:00.321625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:00.326256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:00.428212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9eb2-a15e-4af6-852c-6511634ad737 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:00.461518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:00.462117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:00.462400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:00.462677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:00.463535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:00.463843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:00.465326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:00.482702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 637/1281] 10.222.0.79 () {56 vars in 1143 bytes} [Wed Jul 20 19:18:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:00.488358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:00.577327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:00.587445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abec89-5159-44a9-99eb-ea1049fddf34 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:00.620017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:00.620659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:00.620977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:00.621256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:00.687925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:00.689403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:00.775553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:00.786288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 645/1282] 10.222.0.79 () {58 vars in 1282 bytes} [Wed Jul 20 19:18:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:18:00.791989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:00.854225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:00.896143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc4e8-3833-43f8-99e6-ae69e014d91f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:00.931175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:00.931969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:00.932248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:00.932520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:00.943685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:00.945145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:18:00.963757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:18:00.964854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:18:00.969257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:18:00.969257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:18:00.969257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:18:00.969257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:18:00.971062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:00.972965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 638/1283] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:18:01.080847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298429-4af3-49ae-85c1-231b21cb24b7 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:01.114890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:01.115558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.115843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.116122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.116846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:01.117190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:01.118579 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:01.130635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 646/1284] 10.222.0.79 () {56 vars in 1109 bytes} [Wed Jul 20 19:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:01.136258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:01.238803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1365e8-191f-46c8-acc3-5628aa9f66da None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:01.253295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:01.273593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:01.274377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.274659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.274935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.282078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:01.283508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:18:01.297971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:18:01.299020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 20 19:18:01.300150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 639/1285] 10.222.0.79 () {56 vars in 1111 bytes} [Wed Jul 20 19:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:18:01.306786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:01.357574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:01.413329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fd0f3-4af6-408f-a22b-eaecf6e5e919 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:01.447446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:01.448104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.448384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.448655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.449499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:01.449802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:01.451355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f8ccf-5888-457b-b82d-79954957c450 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:01.463385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 647/1286] 10.222.0.79 () {56 vars in 1148 bytes} [Wed Jul 20 19:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:01.470058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:01.571432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bd801-4d9d-4eee-a5f5-02365203831d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:01.604896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:01.605627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.605904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.606224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.607783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] RBAC: Adding query filter params (role.id=689534f62373409f9d02ea4acc335ab7, scope.project.id=ec730e6d5610427eb8e3d2ae8808c123, user.id=add72a4c345b4ab49e822f478eebd6b8) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:01.608036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:01.609491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478dddf-dc8b-4bb0-9251-bb81725283c4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:01.615497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 640/1287] 10.222.0.79 () {56 vars in 1374 bytes} [Wed Jul 20 19:18:01 2022] GET /identity/v3/role_assignments?user.id=add72a4c345b4ab49e822f478eebd6b8&scope.project.id=ec730e6d5610427eb8e3d2ae8808c123&role.id=689534f62373409f9d02ea4acc335ab7 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:03.682336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaeb98-5c98-477d-9094-1791d9d35c6d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:03.682906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaeb98-5c98-477d-9094-1791d9d35c6d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:03.682906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaeb98-5c98-477d-9094-1791d9d35c6d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:03.684339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 648/1288] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:03.689521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:03.689925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:03.690090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:03.709847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:03.711062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:03.744614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:03.783460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 641/1289] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:18:03.806559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaeb98-5c98-477d-9094-1791d9d35c6d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:03.909621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaeb98-5c98-477d-9094-1791d9d35c6d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:03.921026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:03.944061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:03.944794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:03.945107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:03.945383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:03.946034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:03.947389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:18:03.959852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:18:03.961131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:03.962042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 649/1290] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:18:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:18:03.967658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:04.063827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810a4b3-61da-4ff5-b897-e4f94a89e637 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:04.106429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:04.107370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:04.107655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:04.107928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:04.108633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:04.108969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:04.110331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce619e1-8211-4023-b878-6c771fd16bb0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:04.115867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 642/1291] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Jul 20 19:18:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:04.121551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:04.224283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dd6e6-1b06-40e6-a74a-3bf861fe6e40 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:04.257218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:04.257873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:04.258160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:04.258438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] PATH_INFO: `/v3/services` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:04.259140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:04.259447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:04.260806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e164bb2-80db-4d07-9741-6245d400f4e0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:04.266029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 650/1292] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Jul 20 19:18:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:06.301582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16df7d8-bad0-43cc-87a7-685702782365 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:06.302116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16df7d8-bad0-43cc-87a7-685702782365 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:06.302116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16df7d8-bad0-43cc-87a7-685702782365 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:06.303513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 643/1293] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:06.308539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6bc1b-c9de-4c89-9277-68c81e7051b7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:06.308968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6bc1b-c9de-4c89-9277-68c81e7051b7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:06.309172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6bc1b-c9de-4c89-9277-68c81e7051b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:06.326323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-17a6bc1b-c9de-4c89-9277-68c81e7051b7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:06.327487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-17a6bc1b-c9de-4c89-9277-68c81e7051b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:06.356365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:06.401456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 651/1294] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:18:06.428118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16df7d8-bad0-43cc-87a7-685702782365 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:06.463460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:06.534215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16df7d8-bad0-43cc-87a7-685702782365 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:06.569032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790650ab-e8cd-4a14-a205-675d0289d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:06.569609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790650ab-e8cd-4a14-a205-675d0289d4d9 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:06.569917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790650ab-e8cd-4a14-a205-675d0289d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:06.570247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790650ab-e8cd-4a14-a205-675d0289d4d9 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:06.570984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790650ab-e8cd-4a14-a205-675d0289d4d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:06.572340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790650ab-e8cd-4a14-a205-675d0289d4d9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:06.585877 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 644/1295] 10.222.0.79 () {60 vars in 1141 bytes} [Wed Jul 20 19:18:06 2022] POST /identity/v3/services => generated 246 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:18:08.634461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861050-4713-42b2-8180-37a80b9d3499 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:08.635030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861050-4713-42b2-8180-37a80b9d3499 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:08.635030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861050-4713-42b2-8180-37a80b9d3499 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:08.636401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 652/1296] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:08.642264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:08.642691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:08.642836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:08.660045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:08.661273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:08.734810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 645/1297] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:18:08.757397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53861050-4713-42b2-8180-37a80b9d3499 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:08.857171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53861050-4713-42b2-8180-37a80b9d3499 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:08.892739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:08.893494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:08.893777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:08.894052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:08.894709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:08.896068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:18:08.908819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:18:08.909998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:08.910901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 653/1298] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:18:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:18:08.915986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:08.934875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:09.015750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af1561-bea7-45c2-90e7-2debaaa8da7e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:09.050001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:09.050650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:09.050929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:09.051200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:09.051899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:09.052204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:09.053585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:09.058729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 646/1299] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Jul 20 19:18:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:09.065064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:09.095847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:09.162586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5d9fe-943d-4ede-8795-52708676c9a0 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:09.194586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:09.195224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:09.195503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:09.195776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:09.196549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] RBAC: Adding query filter params (interface=public, service_id=acfb331d89cf41e4a0d0a2aaacd00887, region_id=RegionOne) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:09.196864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:09.198236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba38a-fe3b-45f5-b02c-6a8d8f7e2aaa None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:09.203637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 654/1300] 10.222.0.79 () {56 vars in 1251 bytes} [Wed Jul 20 19:18:09 2022] GET /identity/v3/endpoints?service_id=acfb331d89cf41e4a0d0a2aaacd00887&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:09.209223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:09.290044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:09.307905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07875a2-5615-4745-a637-b8ff52e8339e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:09.341847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb5c3-90e6-4540-be4e-d1bb37d972b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:09.342492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb5c3-90e6-4540-be4e-d1bb37d972b7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:09.342772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb5c3-90e6-4540-be4e-d1bb37d972b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:09.343045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb5c3-90e6-4540-be4e-d1bb37d972b7 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:09.343726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb5c3-90e6-4540-be4e-d1bb37d972b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:09.345107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb5c3-90e6-4540-be4e-d1bb37d972b7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:09.351888 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 647/1301] 10.222.0.79 () {56 vars in 1087 bytes} [Wed Jul 20 19:18:09 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:18:11.380241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a64f9-30be-4ae3-8338-a1a316de2130 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:11.380874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a64f9-30be-4ae3-8338-a1a316de2130 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:11.380874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a64f9-30be-4ae3-8338-a1a316de2130 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:11.382916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 655/1302] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:11.389762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:11.390269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:11.390373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:11.407668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:11.408862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:11.475205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 648/1303] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:18:11.498919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a64f9-30be-4ae3-8338-a1a316de2130 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:11.595842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:11.602854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a64f9-30be-4ae3-8338-a1a316de2130 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:11.638791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:11.639502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:11.639782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:11.640054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:11.640699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:11.642080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:18:11.654062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:18:11.655204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application with Lock( Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: ERROR keystone.server.flask.application Jul 20 19:18:11.656238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 656/1304] 10.222.0.79 () {56 vars in 1117 bytes} [Wed Jul 20 19:18:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:18:11.661221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:11.737205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:11.760457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21850a-482f-4d07-a96e-13f0262c4e9c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:11.794611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:11.795277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:11.795564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:11.795841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:11.796561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:18:11.796901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:11.798271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf707a3-7ba8-44b5-aab6-42c7717a49a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:11.803367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 649/1305] 10.222.0.79 () {56 vars in 1116 bytes} [Wed Jul 20 19:18:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:18:11.810378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:11.908093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8eb92-30c8-4269-acf9-e13ac60b6845 None admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:11.942404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09df04-6942-4cde-be2d-b4634ee81b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:11.943057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09df04-6942-4cde-be2d-b4634ee81b5d None admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:11.943343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09df04-6942-4cde-be2d-b4634ee81b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:11.943618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09df04-6942-4cde-be2d-b4634ee81b5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:11.944356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09df04-6942-4cde-be2d-b4634ee81b5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:11.945745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09df04-6942-4cde-be2d-b4634ee81b5d None admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:11.963490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 657/1306] 10.222.0.79 () {60 vars in 1144 bytes} [Wed Jul 20 19:18:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:18:21.800312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=30 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:21.959434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:16.634465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a346e6f-1132-496b-8d1b-5964e64dcbbd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:16.635150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a346e6f-1132-496b-8d1b-5964e64dcbbd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:16.635150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a346e6f-1132-496b-8d1b-5964e64dcbbd None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:16.642506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebddbc-6736-4229-869d-e61c3a07941d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:16.642934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebddbc-6736-4229-869d-e61c3a07941d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:16.643014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebddbc-6736-4229-869d-e61c3a07941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:16.645222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 650/1307] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:16.804300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a4ebddbc-6736-4229-869d-e61c3a07941d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:16.805934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a4ebddbc-6736-4229-869d-e61c3a07941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:16.940934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 658/1308] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:16.995763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ee5b2-8a89-4ff1-b6f9-bacea5bc11e0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:16.996290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ee5b2-8a89-4ff1-b6f9-bacea5bc11e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:16.996645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ee5b2-8a89-4ff1-b6f9-bacea5bc11e0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:16.998834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 651/1309] 10.222.0.79 () {54 vars in 965 bytes} [Wed Jul 20 19:19:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:17.005680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365deee2-753f-4f2b-9738-a162d27eb55c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:17.006227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365deee2-753f-4f2b-9738-a162d27eb55c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:17.006360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365deee2-753f-4f2b-9738-a162d27eb55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:17.089642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-365deee2-753f-4f2b-9738-a162d27eb55c None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:17.091390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-365deee2-753f-4f2b-9738-a162d27eb55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:17.113603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:17.173762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 659/1310] 10.222.0.79 () {58 vars in 1062 bytes} [Wed Jul 20 19:19:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:17.182836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ee5b2-8a89-4ff1-b6f9-bacea5bc11e0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:17.184794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-131ee5b2-8a89-4ff1-b6f9-bacea5bc11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:17.306017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ee5b2-8a89-4ff1-b6f9-bacea5bc11e0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:17.386542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:17.387184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:17.387473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:17.387732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:17.389558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:17.447332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:17.506536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:17.507965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3b94d-a9b0-4df5-bb3b-f386d8624f1a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:17.530894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 652/1311] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:19.897577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70868f-9656-4142-94f1-327926d3d762 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:19.898129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70868f-9656-4142-94f1-327926d3d762 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:19.898129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70868f-9656-4142-94f1-327926d3d762 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:19.899679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 660/1312] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:19.905629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5ea8-6a20-44f2-b995-777f8fa7d780 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:19.906044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5ea8-6a20-44f2-b995-777f8fa7d780 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:19.906194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5ea8-6a20-44f2-b995-777f8fa7d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:20.011111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c18d5ea8-6a20-44f2-b995-777f8fa7d780 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:20.012379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c18d5ea8-6a20-44f2-b995-777f8fa7d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:20.131036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 653/1313] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:20.171981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70868f-9656-4142-94f1-327926d3d762 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:20.173504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fa70868f-9656-4142-94f1-327926d3d762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:20.289375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70868f-9656-4142-94f1-327926d3d762 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:20.372325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:20.374876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:20.375555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:20.376094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:20.376601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:20.378656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:20.492453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:20.493930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf410e-4264-4d81-9697-59874f81299c service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:20.513063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 661/1314] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:22.913056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1657f6a-9b57-41d7-8f1b-774c199ff610 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:22.913586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1657f6a-9b57-41d7-8f1b-774c199ff610 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:22.913586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1657f6a-9b57-41d7-8f1b-774c199ff610 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:22.915029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 654/1315] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:22.920440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db0dd-fbb3-481b-97c1-672e9047e3ea None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:22.920853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db0dd-fbb3-481b-97c1-672e9047e3ea None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:22.921013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db0dd-fbb3-481b-97c1-672e9047e3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:22.986021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4d3db0dd-fbb3-481b-97c1-672e9047e3ea None None] MFA Rules not processed for user `627b0bc28165465490f31ac21975b10e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:22.987154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4d3db0dd-fbb3-481b-97c1-672e9047e3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:23.041229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:23.086983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 662/1316] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:23.126359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1657f6a-9b57-41d7-8f1b-774c199ff610 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:23.128264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f1657f6a-9b57-41d7-8f1b-774c199ff610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:23.138693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:23.235408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1657f6a-9b57-41d7-8f1b-774c199ff610 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:23.310072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:23.310722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:23.311006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:23.311275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:23.313095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:23.382252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:23.434299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:23.435741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6d585-a63e-456f-8d5d-402dd2e77f05 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:23.458586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 655/1317] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:26.104855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e9c86-13d2-47ec-b386-ba549b7c9f94 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:26.105432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e9c86-13d2-47ec-b386-ba549b7c9f94 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:26.105432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e9c86-13d2-47ec-b386-ba549b7c9f94 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:26.106840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 663/1318] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:26.111983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d85e16-59a6-4dc9-9f57-c3975adde37c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:26.112409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d85e16-59a6-4dc9-9f57-c3975adde37c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:26.112538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d85e16-59a6-4dc9-9f57-c3975adde37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:26.177181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-88d85e16-59a6-4dc9-9f57-c3975adde37c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:26.178412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-88d85e16-59a6-4dc9-9f57-c3975adde37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:26.276703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 656/1319] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:26.316037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e9c86-13d2-47ec-b386-ba549b7c9f94 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:26.317673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-281e9c86-13d2-47ec-b386-ba549b7c9f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:26.425247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e9c86-13d2-47ec-b386-ba549b7c9f94 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:26.499696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:26.500348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:26.500633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:26.500928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:26.502586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:26.510095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:26.611699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:26.613173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114134e-49c1-419e-86ab-2ec1936c8a8e service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:26.631805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 664/1320] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:29.415267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc021005-d4ad-48c8-9dcd-d005dc8069ec None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:29.415879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc021005-d4ad-48c8-9dcd-d005dc8069ec None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:29.415879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc021005-d4ad-48c8-9dcd-d005dc8069ec None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:29.417977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 657/1321] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:29.423228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b81212-d82a-402a-ad4b-1548fdcbd61b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:29.423663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b81212-d82a-402a-ad4b-1548fdcbd61b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:29.423777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b81212-d82a-402a-ad4b-1548fdcbd61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:29.498892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-57b81212-d82a-402a-ad4b-1548fdcbd61b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:29.500131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-57b81212-d82a-402a-ad4b-1548fdcbd61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:29.562201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:29.611927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 665/1322] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:29.650129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc021005-d4ad-48c8-9dcd-d005dc8069ec None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:29.651758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cc021005-d4ad-48c8-9dcd-d005dc8069ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:29.674352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:29.788822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc021005-d4ad-48c8-9dcd-d005dc8069ec None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:29.880175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:29.880830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:29.881145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:29.881423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:29.883101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:29.962731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:30.003520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:30.004859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c19466-6bf1-40e1-84ef-4dc52a98e4ae service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:30.023182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 658/1323] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:32.152821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1b8d2-45ec-4649-86e6-9ed7f9ef4246 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:32.153742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1b8d2-45ec-4649-86e6-9ed7f9ef4246 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:32.153742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1b8d2-45ec-4649-86e6-9ed7f9ef4246 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:32.155218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 666/1324] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:32.161208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672511f-aab1-4c82-b603-19efb372ac3f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:32.161745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672511f-aab1-4c82-b603-19efb372ac3f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:32.161745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672511f-aab1-4c82-b603-19efb372ac3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:32.233039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6672511f-aab1-4c82-b603-19efb372ac3f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:32.234276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6672511f-aab1-4c82-b603-19efb372ac3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:32.358497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 659/1325] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:32.431744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0de484-89b0-492a-bd29-7e172719458a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:32.432323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0de484-89b0-492a-bd29-7e172719458a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:32.432323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0de484-89b0-492a-bd29-7e172719458a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:32.434618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 667/1326] 10.222.0.79 () {54 vars in 965 bytes} [Wed Jul 20 19:19:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:32.442597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3dfc-c40d-456e-a389-411d74bcb930 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:32.442991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3dfc-c40d-456e-a389-411d74bcb930 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:32.443516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3dfc-c40d-456e-a389-411d74bcb930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:32.485166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:32.545069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ce6b3dfc-c40d-456e-a389-411d74bcb930 None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:32.546321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ce6b3dfc-c40d-456e-a389-411d74bcb930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:32.631105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 660/1327] 10.222.0.79 () {58 vars in 1062 bytes} [Wed Jul 20 19:19:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:32.641052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0de484-89b0-492a-bd29-7e172719458a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:32.642703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fe0de484-89b0-492a-bd29-7e172719458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:32.774418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0de484-89b0-492a-bd29-7e172719458a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:32.882186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:32.882842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:32.883127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:32.883398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:32.885122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:32.906124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:33.036657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:33.038188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d4ab9-f582-45a0-b4e1-a963cf6bdc41 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:33.063268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 668/1328] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:35.496457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d91a5a-fe63-49ef-b609-912d7e4e3424 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:35.497430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d91a5a-fe63-49ef-b609-912d7e4e3424 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:35.497919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d91a5a-fe63-49ef-b609-912d7e4e3424 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:35.499573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 661/1329] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:35.505279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f262-48c9-4cca-bf70-26efa3510027 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:35.505765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f262-48c9-4cca-bf70-26efa3510027 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:35.505916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f262-48c9-4cca-bf70-26efa3510027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:35.585528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1445f262-48c9-4cca-bf70-26efa3510027 None None] MFA Rules not processed for user `627b0bc28165465490f31ac21975b10e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:35.586757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1445f262-48c9-4cca-bf70-26efa3510027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:35.652048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:35.687127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 669/1330] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:35.726280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d91a5a-fe63-49ef-b609-912d7e4e3424 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:35.727834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a9d91a5a-fe63-49ef-b609-912d7e4e3424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:35.837660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d91a5a-fe63-49ef-b609-912d7e4e3424 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:35.858245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:35.913522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:35.914165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:35.914445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:35.914717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:35.916342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:36.029519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:36.030973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2c40d-e6c3-4089-beb7-117daa285a57 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:36.049965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 662/1331] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:38.585334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca33c1-baf3-4382-88b5-55de8800d425 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.585798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca33c1-baf3-4382-88b5-55de8800d425 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.585888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca33c1-baf3-4382-88b5-55de8800d425 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.587526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 670/1332] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:38.593532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73520-53cd-4a9e-bea2-2d8947d8903a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.593983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73520-53cd-4a9e-bea2-2d8947d8903a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.594089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73520-53cd-4a9e-bea2-2d8947d8903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.660095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:38.663229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-25b73520-53cd-4a9e-bea2-2d8947d8903a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:38.664869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-25b73520-53cd-4a9e-bea2-2d8947d8903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.766027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 663/1333] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:38.805229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca33c1-baf3-4382-88b5-55de8800d425 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:38.806785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6ca33c1-baf3-4382-88b5-55de8800d425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.910753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca33c1-baf3-4382-88b5-55de8800d425 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:38.986294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:38.986852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.987129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.987401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.989136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:39.027950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:39.110504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:39.111944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8230927-1b74-4cbf-bdcb-59f3aba18149 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:39.129867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 671/1334] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:41.241111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8be2b0-48f1-44e4-b489-71b3e379cb7d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:41.241662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8be2b0-48f1-44e4-b489-71b3e379cb7d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:41.241662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8be2b0-48f1-44e4-b489-71b3e379cb7d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:41.243067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 664/1335] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:41.248738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531e267-6d14-4dd4-8847-a11ff65f5eee None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:41.249152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531e267-6d14-4dd4-8847-a11ff65f5eee None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:41.249337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531e267-6d14-4dd4-8847-a11ff65f5eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:41.320280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6531e267-6d14-4dd4-8847-a11ff65f5eee None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:41.321529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6531e267-6d14-4dd4-8847-a11ff65f5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:41.399317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:41.424243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 672/1336] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:41.465380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8be2b0-48f1-44e4-b489-71b3e379cb7d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:41.466937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ec8be2b0-48f1-44e4-b489-71b3e379cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:41.569176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8be2b0-48f1-44e4-b489-71b3e379cb7d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:41.596348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:41.641609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:41.642258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:41.642571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:41.642849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:41.644510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:41.767018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:41.768455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97889ecf-8c2c-4323-8a42-4232801090d6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:41.786642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 665/1337] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:44.179302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300da9a-db49-4f75-894f-b5c777d63ae7 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:44.179848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300da9a-db49-4f75-894f-b5c777d63ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:44.179848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300da9a-db49-4f75-894f-b5c777d63ae7 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:44.181279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 673/1338] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:44.187117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47a1e-08dd-418a-bd21-a10c667fe535 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:44.187550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47a1e-08dd-418a-bd21-a10c667fe535 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:44.187679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47a1e-08dd-418a-bd21-a10c667fe535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:44.256044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bca47a1e-08dd-418a-bd21-a10c667fe535 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:44.257302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bca47a1e-08dd-418a-bd21-a10c667fe535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:44.267501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:44.368622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 666/1339] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:44.411369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300da9a-db49-4f75-894f-b5c777d63ae7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:44.413607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3300da9a-db49-4f75-894f-b5c777d63ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:44.523611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300da9a-db49-4f75-894f-b5c777d63ae7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:44.600592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:44.601285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:44.601573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:44.601852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:44.603512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:44.640719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:44.719832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:44.721342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cdd56-1722-459f-ba92-dd87fad9c013 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:44.739339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 674/1340] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:49.001219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4325221-670b-4ac4-aeb6-2c8005c849b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:49.001802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4325221-670b-4ac4-aeb6-2c8005c849b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:49.001802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4325221-670b-4ac4-aeb6-2c8005c849b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:49.003164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 667/1341] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:49.008455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c958af-1aa7-417f-87ab-f6842a084e56 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:49.008915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c958af-1aa7-417f-87ab-f6842a084e56 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:49.009052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c958af-1aa7-417f-87ab-f6842a084e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:49.079358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-34c958af-1aa7-417f-87ab-f6842a084e56 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:49.080592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-34c958af-1aa7-417f-87ab-f6842a084e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:49.174011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:49.187073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 675/1342] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:49.232356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4325221-670b-4ac4-aeb6-2c8005c849b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:49.234755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a4325221-670b-4ac4-aeb6-2c8005c849b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:49.346298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4325221-670b-4ac4-aeb6-2c8005c849b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:49.386265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:49.425690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:49.426347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:49.426636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:49.426915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:49.428561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:49.545654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:49.547095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcde0f9-1d89-47fc-8132-cc602b1aaa84 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:49.565688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 668/1343] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:52.355390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201814ed-b7ba-45f6-8eac-a108f907c376 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.355909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201814ed-b7ba-45f6-8eac-a108f907c376 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.355909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201814ed-b7ba-45f6-8eac-a108f907c376 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.357306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 676/1344] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:52.363063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf951-bce7-401c-808d-653f34ec494f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.364214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf951-bce7-401c-808d-653f34ec494f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.364214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf951-bce7-401c-808d-653f34ec494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.463988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ccebf951-bce7-401c-808d-653f34ec494f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:52.465242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ccebf951-bce7-401c-808d-653f34ec494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.484433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:52.569235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 669/1345] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:52.607767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201814ed-b7ba-45f6-8eac-a108f907c376 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:52.609296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-201814ed-b7ba-45f6-8eac-a108f907c376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.730577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201814ed-b7ba-45f6-8eac-a108f907c376 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:52.815075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:52.815733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:52.816018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:52.816297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:52.818030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:52.865499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:52.943611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:52.945099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c893c-9767-4321-adbd-6167e0bafe3e service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:52.964507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 677/1346] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:57.478786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b51b3-70c1-4db0-9907-e2bf3baeaee1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:57.479325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b51b3-70c1-4db0-9907-e2bf3baeaee1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:57.479694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b51b3-70c1-4db0-9907-e2bf3baeaee1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:57.481219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 670/1347] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:57.487059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e2999-d3ca-4c99-9b14-3bcefe940982 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:57.487603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e2999-d3ca-4c99-9b14-3bcefe940982 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:57.487603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e2999-d3ca-4c99-9b14-3bcefe940982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:57.560483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-450e2999-d3ca-4c99-9b14-3bcefe940982 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:57.561756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-450e2999-d3ca-4c99-9b14-3bcefe940982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:57.675770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:57.680469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 678/1348] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:57.720796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b51b3-70c1-4db0-9907-e2bf3baeaee1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:57.722406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-be4b51b3-70c1-4db0-9907-e2bf3baeaee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:57.841005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b51b3-70c1-4db0-9907-e2bf3baeaee1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:57.894406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:57.923999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:57.924664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:57.924980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:57.925343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:57.926986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:58.043092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:58.044543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5370eb8-2de2-4f8b-a2e1-bdff61968511 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:58.064036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 671/1349] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:03.234513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ed885-7cbe-4550-a456-5a1d0a649690 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:03.235099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ed885-7cbe-4550-a456-5a1d0a649690 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:03.235099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ed885-7cbe-4550-a456-5a1d0a649690 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:03.236554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 679/1350] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:03.245371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a8022-bbcf-466b-ab67-b2304e90354c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:03.245895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a8022-bbcf-466b-ab67-b2304e90354c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:03.245983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a8022-bbcf-466b-ab67-b2304e90354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:03.404615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a34a8022-bbcf-466b-ab67-b2304e90354c None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:03.424408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a34a8022-bbcf-466b-ab67-b2304e90354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:03.465774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:03.568827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 672/1351] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:08.066201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccdba7-eeae-4166-9869-5b4a11f27186 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:08.067309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccdba7-eeae-4166-9869-5b4a11f27186 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:08.067832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccdba7-eeae-4166-9869-5b4a11f27186 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:08.069767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 680/1352] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:20:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:08.075540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff095e-933f-49ab-8a58-8a7d127bf94e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:08.076423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff095e-933f-49ab-8a58-8a7d127bf94e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:08.076971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff095e-933f-49ab-8a58-8a7d127bf94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:08.165968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-42ff095e-933f-49ab-8a58-8a7d127bf94e None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:08.167712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-42ff095e-933f-49ab-8a58-8a7d127bf94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:08.262016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:08.313924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 673/1353] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:20:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:09.535026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6012e6-a14c-4332-a10f-f907c2b33398 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:09.535538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6012e6-a14c-4332-a10f-f907c2b33398 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:09.535798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6012e6-a14c-4332-a10f-f907c2b33398 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:09.537310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 681/1354] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:09.543850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d69a2a-07a7-4d3d-8c02-8ad9194bd8f3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:09.544356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d69a2a-07a7-4d3d-8c02-8ad9194bd8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:09.544440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d69a2a-07a7-4d3d-8c02-8ad9194bd8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:09.629266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41d69a2a-07a7-4d3d-8c02-8ad9194bd8f3 None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:09.630470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41d69a2a-07a7-4d3d-8c02-8ad9194bd8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:09.752533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 674/1355] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:20:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:13.294714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf480516-6fc7-4b7b-9b40-ebe0355bfd04 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:13.295292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf480516-6fc7-4b7b-9b40-ebe0355bfd04 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:13.295292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf480516-6fc7-4b7b-9b40-ebe0355bfd04 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:13.296816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 682/1356] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:13.302417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e8c4-e20a-4296-9395-7b90b979994d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:13.302932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e8c4-e20a-4296-9395-7b90b979994d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:13.303032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e8c4-e20a-4296-9395-7b90b979994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:13.358162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:13.416915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-93d2e8c4-e20a-4296-9395-7b90b979994d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:13.416915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-93d2e8c4-e20a-4296-9395-7b90b979994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:13.523998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 675/1357] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:13.600686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21732e-838d-42cb-b980-3baeb5a8d0ce None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:13.601084 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21732e-838d-42cb-b980-3baeb5a8d0ce None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:13.601251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21732e-838d-42cb-b980-3baeb5a8d0ce None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:13.603081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 683/1358] 10.222.0.79 () {54 vars in 962 bytes} [Wed Jul 20 19:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:13.615551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e582c9d-ffe0-4f13-94e1-7805b4f63373 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:13.616824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e582c9d-ffe0-4f13-94e1-7805b4f63373 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:13.618114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e582c9d-ffe0-4f13-94e1-7805b4f63373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:13.708089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8e582c9d-ffe0-4f13-94e1-7805b4f63373 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:13.709455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e582c9d-ffe0-4f13-94e1-7805b4f63373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:13.771990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:13.850277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 676/1359] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Jul 20 19:20:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:13.858807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21732e-838d-42cb-b980-3baeb5a8d0ce None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:13.860319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8b21732e-838d-42cb-b980-3baeb5a8d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:14.017591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21732e-838d-42cb-b980-3baeb5a8d0ce None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:14.098028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:14.144612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:14.145341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:14.145631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:14.145905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:14.147547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:14.245238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:14.246832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b604960-10e0-4e19-bf02-76cb460bddc0 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:14.266747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 684/1360] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:16.656166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f526-99e0-4c1f-adbb-4f8b11f27ae9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:16.656714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f526-99e0-4c1f-adbb-4f8b11f27ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:16.657212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f526-99e0-4c1f-adbb-4f8b11f27ae9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:16.659620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 677/1361] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:16.668300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630da51-c212-4eae-ac55-5146409fe455 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:16.668783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630da51-c212-4eae-ac55-5146409fe455 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:16.669216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630da51-c212-4eae-ac55-5146409fe455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:16.774406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1630da51-c212-4eae-ac55-5146409fe455 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:16.776298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1630da51-c212-4eae-ac55-5146409fe455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:16.816229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:16.896550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 685/1362] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:16.943817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506f526-99e0-4c1f-adbb-4f8b11f27ae9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:16.945396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b506f526-99e0-4c1f-adbb-4f8b11f27ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:17.122803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506f526-99e0-4c1f-adbb-4f8b11f27ae9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:17.133419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:17.261280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:17.261937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:17.262228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:17.262507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:17.264191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:17.359752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:17.361569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53aaba7-a917-4d99-9336-ff7ef38cff48 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:17.380946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 678/1363] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:19.809373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be713-677a-4042-be75-b62d71c3b997 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:19.809933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be713-677a-4042-be75-b62d71c3b997 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:19.809933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853be713-677a-4042-be75-b62d71c3b997 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:19.811324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 686/1364] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:19.817018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521bb59-5f15-4b07-9306-8e426271eca1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:19.817475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521bb59-5f15-4b07-9306-8e426271eca1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:19.817869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521bb59-5f15-4b07-9306-8e426271eca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:19.835519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:19.894515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1521bb59-5f15-4b07-9306-8e426271eca1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:19.895878 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1521bb59-5f15-4b07-9306-8e426271eca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:20.005241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 679/1365] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:20.054497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7fbc1-1ab7-40b8-a097-7b6e2a6b79a9 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:20.054955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7fbc1-1ab7-40b8-a097-7b6e2a6b79a9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:20.055706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7fbc1-1ab7-40b8-a097-7b6e2a6b79a9 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:20.059578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 687/1366] 10.222.0.79 () {54 vars in 962 bytes} [Wed Jul 20 19:20:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:20.065557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef791-4fc8-4e55-8377-f25352400285 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:20.065850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef791-4fc8-4e55-8377-f25352400285 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:20.066271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef791-4fc8-4e55-8377-f25352400285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:20.151949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ff1ef791-4fc8-4e55-8377-f25352400285 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:20.153445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ff1ef791-4fc8-4e55-8377-f25352400285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:20.191051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:20.285950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 680/1367] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Jul 20 19:20:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:20.293688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7fbc1-1ab7-40b8-a097-7b6e2a6b79a9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:20.295210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-acc7fbc1-1ab7-40b8-a097-7b6e2a6b79a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:20.461659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7fbc1-1ab7-40b8-a097-7b6e2a6b79a9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:20.484908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:20.586396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:20.586995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:20.587282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:20.587627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:20.589848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:20.692412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:20.694135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39bf76-2ac4-4e65-a85b-2b6b08dc89af service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:20.716944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 688/1368] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:21.714190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117198a-bc76-4eef-bc49-a422d4b69ee7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:21.714653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117198a-bc76-4eef-bc49-a422d4b69ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:21.714778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117198a-bc76-4eef-bc49-a422d4b69ee7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:21.716300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 681/1369] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:21.721333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f78f97-6e47-4b41-bf03-a9159018e288 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:21.721729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f78f97-6e47-4b41-bf03-a9159018e288 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:21.721913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f78f97-6e47-4b41-bf03-a9159018e288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:21.754539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:21.792330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-64f78f97-6e47-4b41-bf03-a9159018e288 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:21.793581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-64f78f97-6e47-4b41-bf03-a9159018e288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:21.894925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 689/1370] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:22.253363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f71909-e26b-4cfc-af8c-e4bfed152485 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:22.253363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f71909-e26b-4cfc-af8c-e4bfed152485 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:22.253363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f71909-e26b-4cfc-af8c-e4bfed152485 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:22.253363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 682/1371] 10.222.0.79 () {54 vars in 964 bytes} [Wed Jul 20 19:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:22.259933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541729e-81dc-4aac-8a1c-ba73d8b011e2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:22.260597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541729e-81dc-4aac-8a1c-ba73d8b011e2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:22.261039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541729e-81dc-4aac-8a1c-ba73d8b011e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:22.341968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6541729e-81dc-4aac-8a1c-ba73d8b011e2 None None] MFA Rules not processed for user `69dd61877f6e41ba86d2a9bb9537afc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:22.343211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6541729e-81dc-4aac-8a1c-ba73d8b011e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:22.394176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:22.428546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 690/1372] 10.222.0.79 () {58 vars in 1061 bytes} [Wed Jul 20 19:20:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:22.436684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f71909-e26b-4cfc-af8c-e4bfed152485 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:22.438255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45f71909-e26b-4cfc-af8c-e4bfed152485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:22.553993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f71909-e26b-4cfc-af8c-e4bfed152485 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:22.602315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:22.642377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:22.642937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:22.643216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:22.643489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:22.645164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:22.777856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:22.780550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e199bb-7564-4770-a1bf-8e1d2349eb5d service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:22.810587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 683/1373] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:22.883948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb65c7e-b164-4149-af3e-2c2b78ff5111 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:22.885087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb65c7e-b164-4149-af3e-2c2b78ff5111 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:22.885628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb65c7e-b164-4149-af3e-2c2b78ff5111 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:22.887625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 691/1374] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:22.893295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e523b-b354-4817-960d-d30328cdd00d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:22.893705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e523b-b354-4817-960d-d30328cdd00d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:22.894224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e523b-b354-4817-960d-d30328cdd00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:22.974542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fc8e523b-b354-4817-960d-d30328cdd00d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:22.976191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fc8e523b-b354-4817-960d-d30328cdd00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:23.003230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:23.097475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 684/1375] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:23.143126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb65c7e-b164-4149-af3e-2c2b78ff5111 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:23.144855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7cb65c7e-b164-4149-af3e-2c2b78ff5111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:23.295643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb65c7e-b164-4149-af3e-2c2b78ff5111 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:23.351375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:23.422889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:23.423545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:23.423829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:23.424105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:23.425798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:23.520036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:23.521369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c20e83-6ad6-4224-b756-6c986a03e819 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:23.539707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 692/1376] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:25.717701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ebb3a-abb7-42d6-98e7-d547384f6bdc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:25.718273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ebb3a-abb7-42d6-98e7-d547384f6bdc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:25.718273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ebb3a-abb7-42d6-98e7-d547384f6bdc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:25.719630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 685/1377] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:25.725075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a217e7-ffa4-422c-bfdf-5e5e73e984c7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:25.725531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a217e7-ffa4-422c-bfdf-5e5e73e984c7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:25.725642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a217e7-ffa4-422c-bfdf-5e5e73e984c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:25.796491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-85a217e7-ffa4-422c-bfdf-5e5e73e984c7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:25.797749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-85a217e7-ffa4-422c-bfdf-5e5e73e984c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:25.804861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:25.922503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 693/1378] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:25.972398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ebb3a-abb7-42d6-98e7-d547384f6bdc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:25.974116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef4ebb3a-abb7-42d6-98e7-d547384f6bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:26.165629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ebb3a-abb7-42d6-98e7-d547384f6bdc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:26.170087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:26.285116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:26.285772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:26.286055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:26.286328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:26.288009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:26.383934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:26.385827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feecb7-c95e-4a24-8b4f-4e865b20c795 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:26.409545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 686/1379] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:28.527542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe689c-e615-43b9-aaeb-e19e9c240213 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:28.528075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe689c-e615-43b9-aaeb-e19e9c240213 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:28.528355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe689c-e615-43b9-aaeb-e19e9c240213 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:28.530314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 694/1380] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:28.535556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9364d-83d5-4e53-a5f4-11660cb16a23 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:28.535996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9364d-83d5-4e53-a5f4-11660cb16a23 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:28.536166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9364d-83d5-4e53-a5f4-11660cb16a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:28.547448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:28.607746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-15d9364d-83d5-4e53-a5f4-11660cb16a23 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:28.608983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-15d9364d-83d5-4e53-a5f4-11660cb16a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:28.738899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 687/1381] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:28.783879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe689c-e615-43b9-aaeb-e19e9c240213 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:28.785672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-90fe689c-e615-43b9-aaeb-e19e9c240213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:28.940680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:28.956870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe689c-e615-43b9-aaeb-e19e9c240213 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:29.091040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:29.091613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:29.091902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:29.092180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:29.094059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:29.201702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:29.203940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7271fc3-b929-48e6-b578-373d29c336c9 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:29.218466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:29.229307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 695/1382] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:31.406780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be574f-bf3c-4c77-8701-7a71704f4fc0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:31.407353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be574f-bf3c-4c77-8701-7a71704f4fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:31.407722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be574f-bf3c-4c77-8701-7a71704f4fc0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:31.409621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 688/1383] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:31.415116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e8e3-c7bb-4edb-b399-efe3ad1abe01 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:31.416471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e8e3-c7bb-4edb-b399-efe3ad1abe01 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:31.416905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4e8e3-c7bb-4edb-b399-efe3ad1abe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:31.484451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6dd4e8e3-c7bb-4edb-b399-efe3ad1abe01 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:31.485788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6dd4e8e3-c7bb-4edb-b399-efe3ad1abe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:31.585835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 696/1384] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:31.631101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be574f-bf3c-4c77-8701-7a71704f4fc0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:31.632634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f0be574f-bf3c-4c77-8701-7a71704f4fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:31.731709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:31.812233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be574f-bf3c-4c77-8701-7a71704f4fc0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:31.934438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:31.935095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:31.935380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:31.935658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:31.937661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:31.990706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:32.041247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:32.043260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e7211-43d4-4087-9a2e-781e7294d745 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:32.067398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 689/1385] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:34.155530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230f4b3-09f1-40e0-9192-321493df03b3 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:34.156092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230f4b3-09f1-40e0-9192-321493df03b3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:34.156233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230f4b3-09f1-40e0-9192-321493df03b3 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:34.158117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 697/1386] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:34.164923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22bdf-0abe-4f5f-8b24-72eed7c9e906 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:34.165334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22bdf-0abe-4f5f-8b24-72eed7c9e906 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:34.166430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22bdf-0abe-4f5f-8b24-72eed7c9e906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:34.251399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-88a22bdf-0abe-4f5f-8b24-72eed7c9e906 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:34.252743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-88a22bdf-0abe-4f5f-8b24-72eed7c9e906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:34.394611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 690/1387] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:34.464757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230f4b3-09f1-40e0-9192-321493df03b3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:34.466377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d230f4b3-09f1-40e0-9192-321493df03b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:34.566557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:34.661183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230f4b3-09f1-40e0-9192-321493df03b3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:34.816441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:34.817140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:34.817426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:34.817698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:34.819394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:34.872389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:34.944136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:34.947606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7cbc2-339c-4a34-87b9-72312d36d5e1 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:34.976680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 698/1388] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:37.328307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5766bc-1a85-4133-9581-acc31347dcd0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:37.328872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5766bc-1a85-4133-9581-acc31347dcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:37.329362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5766bc-1a85-4133-9581-acc31347dcd0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:37.331228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 691/1389] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:37.336711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbed87-275c-463c-8654-7432d3916a9a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:37.337737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbed87-275c-463c-8654-7432d3916a9a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:37.337901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbed87-275c-463c-8654-7432d3916a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:37.408528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d8cbed87-275c-463c-8654-7432d3916a9a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:37.409790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d8cbed87-275c-463c-8654-7432d3916a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:37.506000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 699/1390] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:20:37.551711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5766bc-1a85-4133-9581-acc31347dcd0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:37.553263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba5766bc-1a85-4133-9581-acc31347dcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:37.585015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:37.706551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5766bc-1a85-4133-9581-acc31347dcd0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:37.817018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:37.829635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:37.830291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:37.830579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:37.830850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:37.832497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:37.928703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:37.930286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c368885-8b59-47da-b73e-fb9db19b0dc4 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:37.949270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 692/1391] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:38.260993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41ae9-9b54-41c1-8bc0-5969807908d4 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:38.261364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41ae9-9b54-41c1-8bc0-5969807908d4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:38.261646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41ae9-9b54-41c1-8bc0-5969807908d4 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:38.263042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 700/1392] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:38.268452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dda9f3-a796-4ef7-9d2b-78f224ddd263 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:38.268747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dda9f3-a796-4ef7-9d2b-78f224ddd263 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:38.269062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dda9f3-a796-4ef7-9d2b-78f224ddd263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:38.337390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-34dda9f3-a796-4ef7-9d2b-78f224ddd263 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:38.338755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-34dda9f3-a796-4ef7-9d2b-78f224ddd263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:38.399306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:38.438734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 693/1393] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:38.496355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80bb31-0472-446f-9af0-c3c6d1746175 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:38.496836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80bb31-0472-446f-9af0-c3c6d1746175 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:38.496971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80bb31-0472-446f-9af0-c3c6d1746175 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:38.498341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 701/1394] 10.222.0.79 () {54 vars in 966 bytes} [Wed Jul 20 19:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:38.504097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6d77-5dbd-4885-b0c2-6eaa500971d1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:38.504494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6d77-5dbd-4885-b0c2-6eaa500971d1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:38.504846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6d77-5dbd-4885-b0c2-6eaa500971d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:38.582820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5a0a6d77-5dbd-4885-b0c2-6eaa500971d1 None None] MFA Rules not processed for user `cd4e27030ec042bca0c7d301998e018e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:38.584020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5a0a6d77-5dbd-4885-b0c2-6eaa500971d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:38.663323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 694/1395] 10.222.0.79 () {58 vars in 1063 bytes} [Wed Jul 20 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:38.671324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80bb31-0472-446f-9af0-c3c6d1746175 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:38.672845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cd80bb31-0472-446f-9af0-c3c6d1746175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:38.689491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:38.784654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80bb31-0472-446f-9af0-c3c6d1746175 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:38.860754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:38.861349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:38.861639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:38.861912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:38.863565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:38.933254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:38.975073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:38.976527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5bcfa-6bc6-4e2b-8b7d-290ea68342d2 service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:38.996069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 702/1396] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:39.721320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c3efd-0499-4e16-9dea-07c30d18c3d2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:39.721812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c3efd-0499-4e16-9dea-07c30d18c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:39.721812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c3efd-0499-4e16-9dea-07c30d18c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:39.794823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e10c3efd-0499-4e16-9dea-07c30d18c3d2 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:39.796045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e10c3efd-0499-4e16-9dea-07c30d18c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:39.806539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:39.889722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 695/1397] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:20:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:39.932453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bba12-a905-482e-a1fc-bfc1112cfe8f None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:39.933029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bba12-a905-482e-a1fc-bfc1112cfe8f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:39.933183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bba12-a905-482e-a1fc-bfc1112cfe8f None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:39.934668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 703/1398] 10.222.0.79 () {54 vars in 943 bytes} [Wed Jul 20 19:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:39.941327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:39.941717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:39.942058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:40.018315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] MFA Rules not processed for user `04be59c2cfa049629282356a9e209900`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:40.020505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:40.063983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cdff8-2951-49f1-9580-d5db6585ba94 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:40.064273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cdff8-2951-49f1-9580-d5db6585ba94 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:40.064564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cdff8-2951-49f1-9580-d5db6585ba94 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:40.065989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 704/1399] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:40.070909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:40.071200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:40.071488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:40.108150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:40.131929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 696/1400] 10.222.0.79 () {58 vars in 1040 bytes} [Wed Jul 20 19:20:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:40.141704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:40.142533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:40.143756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:40.153798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c0631-c34d-4f7b-a5d7-ffe64a6715c9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:40.251856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 705/1401] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:20:40.271424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:40.272152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:40.272432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:40.272700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:40.274447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:40.297415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:40.298634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:40.326413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:40.367328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:40.368997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e0da1-48a8-484d-b81b-a32d975dd0ca service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:40.377472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 697/1402] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:20:40.419116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c11576-2c7a-4219-a245-fc2140861ea0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:40.559910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:40.560570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:40.560866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:40.561169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:40.562842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:40.583915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:40.697241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:40.697602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f8a54-74c7-4a6e-9897-89267035031c service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:40.735598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 706/1403] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:20:42.918988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0d83a-9ed2-471a-b01f-aabbae182319 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:42.919600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0d83a-9ed2-471a-b01f-aabbae182319 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:42.919600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0d83a-9ed2-471a-b01f-aabbae182319 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:42.920934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 698/1404] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:42.926628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f54613-ff82-4af9-88bc-d2f5690e6bb8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:42.927129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f54613-ff82-4af9-88bc-d2f5690e6bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:42.927316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f54613-ff82-4af9-88bc-d2f5690e6bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:43.001570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-12f54613-ff82-4af9-88bc-d2f5690e6bb8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:43.002808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-12f54613-ff82-4af9-88bc-d2f5690e6bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:43.121426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:43.128429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 707/1405] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:43.173614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0d83a-9ed2-471a-b01f-aabbae182319 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:43.175058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a9d0d83a-9ed2-471a-b01f-aabbae182319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:43.188316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:43.353196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0d83a-9ed2-471a-b01f-aabbae182319 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:43.459654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:43.499458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:43.500245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:43.500676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:43.500953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:43.502946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:43.615329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:43.617163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029009c-853b-4afc-b531-ab4fd1077432 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:43.638749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 699/1406] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:20:45.845105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ebfb-6140-45e8-8c09-b86c0285c405 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:45.845105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ebfb-6140-45e8-8c09-b86c0285c405 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:45.845105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ebfb-6140-45e8-8c09-b86c0285c405 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:45.845105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 708/1407] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:20:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:20:45.845891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34567ea-b08f-41b8-8f1a-8e4227d2d593 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:45.845891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34567ea-b08f-41b8-8f1a-8e4227d2d593 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:45.845891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34567ea-b08f-41b8-8f1a-8e4227d2d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:45.958478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d34567ea-b08f-41b8-8f1a-8e4227d2d593 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:20:45.959630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d34567ea-b08f-41b8-8f1a-8e4227d2d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:46.042524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:46.112964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 700/1408] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:20:46.152524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ebfb-6140-45e8-8c09-b86c0285c405 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:20:46.154195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-53c5ebfb-6140-45e8-8c09-b86c0285c405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:46.333841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ebfb-6140-45e8-8c09-b86c0285c405 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:20:46.434054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:46.509743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:20:46.510416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:20:46.510702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:20:46.510975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:20:46.512760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:20:46.626223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:20:46.629311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09162e-83f0-4729-90f9-e68f5014817d service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:20:46.654986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 709/1409] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:20:56.103667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=23 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:20:56.648850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=64 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:03.391943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697ccaa-5dd8-42b7-b41e-0a1c02197611 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:03.392512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697ccaa-5dd8-42b7-b41e-0a1c02197611 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:03.392512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697ccaa-5dd8-42b7-b41e-0a1c02197611 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:03.393918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 701/1410] 10.222.0.79 () {54 vars in 918 bytes} [Wed Jul 20 19:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:03.399173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ef338-ccd3-4316-85f2-9289f7226d01 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:03.399957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ef338-ccd3-4316-85f2-9289f7226d01 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:03.400515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ef338-ccd3-4316-85f2-9289f7226d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:03.478326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bb3ef338-ccd3-4316-85f2-9289f7226d01 None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:03.479561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bb3ef338-ccd3-4316-85f2-9289f7226d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:03.588667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 710/1411] 10.222.0.79 () {58 vars in 1015 bytes} [Wed Jul 20 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:03.603095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3d1e8-6af4-4dc0-8a14-3770591238cb None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:03.603513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3d1e8-6af4-4dc0-8a14-3770591238cb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:03.603804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3d1e8-6af4-4dc0-8a14-3770591238cb None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:03.605241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 702/1412] 10.222.0.79 () {54 vars in 918 bytes} [Wed Jul 20 19:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:03.609897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25e9b5-68dc-4953-aa77-8b8d165797b7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:03.610188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25e9b5-68dc-4953-aa77-8b8d165797b7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:03.610490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25e9b5-68dc-4953-aa77-8b8d165797b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:03.682629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:03.685689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1b25e9b5-68dc-4953-aa77-8b8d165797b7 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:03.688060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1b25e9b5-68dc-4953-aa77-8b8d165797b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:03.806055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 711/1413] 10.222.0.79 () {58 vars in 1015 bytes} [Wed Jul 20 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:06.367863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d6344-4917-49f3-9934-e2745aa47bd0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:06.368772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d6344-4917-49f3-9934-e2745aa47bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:06.368985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d6344-4917-49f3-9934-e2745aa47bd0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:06.370469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:21:06.376393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd1f2e-5911-43f1-8d46-b9d548d87bb9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:06.376851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd1f2e-5911-43f1-8d46-b9d548d87bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:06.377032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd1f2e-5911-43f1-8d46-b9d548d87bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:06.443719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d4fd1f2e-5911-43f1-8d46-b9d548d87bb9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:06.444970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d4fd1f2e-5911-43f1-8d46-b9d548d87bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:06.501090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:06.544903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:06.589339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d6344-4917-49f3-9934-e2745aa47bd0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:06.590935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5d5d6344-4917-49f3-9934-e2745aa47bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:06.766676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d6344-4917-49f3-9934-e2745aa47bd0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:06.840109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:06.882461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:06.883116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:06.883403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:06.884394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:06.886160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:06.976780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:06.978880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df7750-9cf4-4fbd-bec2-74359b102471 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:06.997521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 704/1416] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:07.916739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3268-20d8-4423-95f2-63d5394355e2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:07.917316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3268-20d8-4423-95f2-63d5394355e2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:07.917316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3268-20d8-4423-95f2-63d5394355e2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:07.918707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:07.923831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77b162-93a5-4db8-9e37-33e5cb8216d6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:07.924267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77b162-93a5-4db8-9e37-33e5cb8216d6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:07.924469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77b162-93a5-4db8-9e37-33e5cb8216d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:08.007670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1d77b162-93a5-4db8-9e37-33e5cb8216d6 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:08.008949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1d77b162-93a5-4db8-9e37-33e5cb8216d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:08.037795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:08.118817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:08.272058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3268-20d8-4423-95f2-63d5394355e2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:08.273539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-398c3268-20d8-4423-95f2-63d5394355e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:08.426059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3268-20d8-4423-95f2-63d5394355e2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:08.467086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:08.540288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:08.540976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:08.541268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:08.541549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:08.543229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:08.631592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:08.633235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58d189-fb7e-4fa9-8d77-7e37000f7e22 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:08.651829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 714/1419] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:09.542489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fff7e-60db-4d8d-adb3-8f5bd7aa7efe None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:09.543078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fff7e-60db-4d8d-adb3-8f5bd7aa7efe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:09.543078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fff7e-60db-4d8d-adb3-8f5bd7aa7efe None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:09.544441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:09.549411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57305390-a46a-477e-b7c8-c54ce023e089 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:09.549815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57305390-a46a-477e-b7c8-c54ce023e089 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:09.550258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57305390-a46a-477e-b7c8-c54ce023e089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:09.604377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:09.627366 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-57305390-a46a-477e-b7c8-c54ce023e089 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:09.628580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-57305390-a46a-477e-b7c8-c54ce023e089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:09.733602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:09.887009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fff7e-60db-4d8d-adb3-8f5bd7aa7efe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:09.888468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d2fff7e-60db-4d8d-adb3-8f5bd7aa7efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:10.040677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fff7e-60db-4d8d-adb3-8f5bd7aa7efe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:10.054065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:10.162406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:10.162953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:10.163241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:10.163520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:10.165901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:10.257520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:10.260013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9ce34-c25d-4f3d-8f05-f56e8e24e2f8 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:10.282148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 707/1422] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:11.152918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79626a8-3a79-4cf6-b256-a5cbb90aa64b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:11.153465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79626a8-3a79-4cf6-b256-a5cbb90aa64b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:11.153559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79626a8-3a79-4cf6-b256-a5cbb90aa64b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:11.155136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:11.160239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c73dd-f8fa-41b1-bdfb-114f63ae2708 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:11.160681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c73dd-f8fa-41b1-bdfb-114f63ae2708 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:11.160791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c73dd-f8fa-41b1-bdfb-114f63ae2708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:11.180911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:11.241220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a68c73dd-f8fa-41b1-bdfb-114f63ae2708 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:11.242454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a68c73dd-f8fa-41b1-bdfb-114f63ae2708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:11.341352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:11.492708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79626a8-3a79-4cf6-b256-a5cbb90aa64b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:11.494952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e79626a8-3a79-4cf6-b256-a5cbb90aa64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:11.608865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:11.645353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79626a8-3a79-4cf6-b256-a5cbb90aa64b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:11.767426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:11.768011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:11.768316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:11.768587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:11.770314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:11.848587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:11.864781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:11.866407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94905c32-3874-46d8-b4c4-e23c283e6fcb service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:11.885241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 717/1425] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:12.771454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f279b3-c4d8-4fa1-8d40-2451de81f8de None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:12.772028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f279b3-c4d8-4fa1-8d40-2451de81f8de None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:12.772028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f279b3-c4d8-4fa1-8d40-2451de81f8de None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:12.773368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:12.779450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99841b7a-5f05-4909-8185-f8ed70fbb62d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:12.780195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99841b7a-5f05-4909-8185-f8ed70fbb62d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:12.780300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99841b7a-5f05-4909-8185-f8ed70fbb62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:12.857600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-99841b7a-5f05-4909-8185-f8ed70fbb62d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:12.858822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-99841b7a-5f05-4909-8185-f8ed70fbb62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:12.959298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:13.111233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f279b3-c4d8-4fa1-8d40-2451de81f8de None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:13.112850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-13f279b3-c4d8-4fa1-8d40-2451de81f8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:13.200636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:13.266616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f279b3-c4d8-4fa1-8d40-2451de81f8de None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:13.387290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:13.387859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:13.388139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:13.388411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:13.390095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:13.441689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:13.484995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:13.486615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e98b-20a1-4199-9b3b-cb53f283846f service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:13.505454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 710/1428] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:14.337689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed4be6-073b-40f2-9cf7-ae0baf3dff41 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:14.338266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed4be6-073b-40f2-9cf7-ae0baf3dff41 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:14.338266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed4be6-073b-40f2-9cf7-ae0baf3dff41 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:14.339751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:14.344979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31f784-d971-45ea-ace0-9e0eed5dd005 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:14.345460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31f784-d971-45ea-ace0-9e0eed5dd005 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:14.345572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31f784-d971-45ea-ace0-9e0eed5dd005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:14.421572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ee31f784-d971-45ea-ace0-9e0eed5dd005 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:14.422806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ee31f784-d971-45ea-ace0-9e0eed5dd005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:14.522558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:21:14.674693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed4be6-073b-40f2-9cf7-ae0baf3dff41 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:14.676146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-92ed4be6-073b-40f2-9cf7-ae0baf3dff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:14.732028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:14.841805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed4be6-073b-40f2-9cf7-ae0baf3dff41 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:14.956620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:14.957220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:14.957512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:14.957795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:14.959540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:14.967266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:15.053610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:15.055296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7db3dd-8149-4a62-8cf6-b4bb2716fd5d service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:15.074270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 720/1431] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:15.896780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1af33-ee1f-464b-8f5f-03f67dfe1290 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:15.897379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1af33-ee1f-464b-8f5f-03f67dfe1290 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:15.897379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1af33-ee1f-464b-8f5f-03f67dfe1290 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:15.898760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:15.904282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d80cd-f8bc-4d7e-bfa0-fc1653f0607a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:15.905092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d80cd-f8bc-4d7e-bfa0-fc1653f0607a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:15.905404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d80cd-f8bc-4d7e-bfa0-fc1653f0607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:15.990797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8a2d80cd-f8bc-4d7e-bfa0-fc1653f0607a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:15.991926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8a2d80cd-f8bc-4d7e-bfa0-fc1653f0607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:16.067450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:16.107066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:16.263387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1af33-ee1f-464b-8f5f-03f67dfe1290 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:16.264868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c6c1af33-ee1f-464b-8f5f-03f67dfe1290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:16.282884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:16.419596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1af33-ee1f-464b-8f5f-03f67dfe1290 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:16.509664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:16.537908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:16.538563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:16.538843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:16.539114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:16.540819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:16.632799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:16.635197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc291b-60b8-4ebe-85ff-bb7d07024fd7 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:16.653017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 713/1434] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:17.478849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9ccf6-92f7-4d78-b581-ac39071be0b0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:17.479429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9ccf6-92f7-4d78-b581-ac39071be0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:17.479429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9ccf6-92f7-4d78-b581-ac39071be0b0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:17.480833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:17.486949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7735f4-a8cf-4c30-9a04-73e416917984 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:17.487478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7735f4-a8cf-4c30-9a04-73e416917984 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:17.487583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7735f4-a8cf-4c30-9a04-73e416917984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:17.572369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ca7735f4-a8cf-4c30-9a04-73e416917984 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:17.573651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca7735f4-a8cf-4c30-9a04-73e416917984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:17.625739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:17.683597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:17.838875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9ccf6-92f7-4d78-b581-ac39071be0b0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:17.840412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e4b9ccf6-92f7-4d78-b581-ac39071be0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:17.998567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9ccf6-92f7-4d78-b581-ac39071be0b0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:18.056440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:18.118513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:18.119269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:18.119583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:18.119860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:18.121610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:18.217172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:18.219020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3435de-db3d-4caa-8fcb-ccb9c88900c9 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:18.240475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 723/1437] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:19.074072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b59432-0efa-41c2-835e-d00400a1cbea None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:19.074666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b59432-0efa-41c2-835e-d00400a1cbea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:19.074666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b59432-0efa-41c2-835e-d00400a1cbea None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:19.076006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:21:19.081090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191eefd-9980-44bc-8c25-8e7f5d234cbd None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:19.081627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191eefd-9980-44bc-8c25-8e7f5d234cbd None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:19.081742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191eefd-9980-44bc-8c25-8e7f5d234cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:19.161354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7191eefd-9980-44bc-8c25-8e7f5d234cbd None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:19.162590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7191eefd-9980-44bc-8c25-8e7f5d234cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:19.164744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:19.262249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:19.414517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b59432-0efa-41c2-835e-d00400a1cbea None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:19.416165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f5b59432-0efa-41c2-835e-d00400a1cbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:19.568825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b59432-0efa-41c2-835e-d00400a1cbea None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:19.597952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:19.686328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:19.686908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:19.687191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:19.687467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:19.689221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:19.789093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:19.790879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2026b-1601-44ae-aee2-61a463c9046b service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:19.809635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 716/1440] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:21.852852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8496a2-eb6a-4a05-8e1f-ada513e34ae5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:21.853452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8496a2-eb6a-4a05-8e1f-ada513e34ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:21.853559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8496a2-eb6a-4a05-8e1f-ada513e34ae5 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:21.855829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:21.861495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62959dfa-c3d2-41fe-841f-fa27b34ac2a1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:21.861967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62959dfa-c3d2-41fe-841f-fa27b34ac2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:21.862078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62959dfa-c3d2-41fe-841f-fa27b34ac2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:21.905535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:21.932147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-62959dfa-c3d2-41fe-841f-fa27b34ac2a1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:21.933485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-62959dfa-c3d2-41fe-841f-fa27b34ac2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:22.043459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:22.084794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8496a2-eb6a-4a05-8e1f-ada513e34ae5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:22.087317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dc8496a2-eb6a-4a05-8e1f-ada513e34ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:22.199688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8496a2-eb6a-4a05-8e1f-ada513e34ae5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:22.233322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:22.276605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:22.277312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:22.277595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:22.277868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:22.279539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:22.397836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:22.399464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13ed3-f100-44c5-9161-2f240c348700 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:22.424486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 726/1443] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:21:24.709796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181af7-6b92-4015-97f3-e234d183a164 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:24.710290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181af7-6b92-4015-97f3-e234d183a164 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:24.710388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181af7-6b92-4015-97f3-e234d183a164 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:24.711785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:24.717673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc505fc-a88d-49e6-a8fb-28e5395c753f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:24.718098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc505fc-a88d-49e6-a8fb-28e5395c753f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:24.718238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc505fc-a88d-49e6-a8fb-28e5395c753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:24.785001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4cc505fc-a88d-49e6-a8fb-28e5395c753f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:24.786219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4cc505fc-a88d-49e6-a8fb-28e5395c753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:24.800156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:24.887928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:24.927965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f181af7-6b92-4015-97f3-e234d183a164 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:24.929611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1f181af7-6b92-4015-97f3-e234d183a164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:25.036018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f181af7-6b92-4015-97f3-e234d183a164 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:25.048461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:25.110837 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:25.111486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:25.111765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:25.112040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:25.113757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:25.227864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:25.229356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9726cfa-015f-4476-aa41-5c4397ef233b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:25.252075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 719/1446] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:27.638713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0d802-a1c2-4b51-b324-7815e5a4cd1d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:27.639318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0d802-a1c2-4b51-b324-7815e5a4cd1d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:27.639318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0d802-a1c2-4b51-b324-7815e5a4cd1d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:27.640672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:27.646329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4abeb-6ae4-4966-bd84-034c4a9defa9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:27.646781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4abeb-6ae4-4966-bd84-034c4a9defa9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:27.646901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4abeb-6ae4-4966-bd84-034c4a9defa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:27.710541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:27.733747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0ef4abeb-6ae4-4966-bd84-034c4a9defa9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:27.734949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ef4abeb-6ae4-4966-bd84-034c4a9defa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:27.839699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:27.879152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0d802-a1c2-4b51-b324-7815e5a4cd1d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:27.880744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f2a0d802-a1c2-4b51-b324-7815e5a4cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:27.989327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0d802-a1c2-4b51-b324-7815e5a4cd1d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:28.033223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:28.068956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:28.069613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:28.070546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:28.070546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:28.072570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:28.183961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:28.185451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f210e8-896e-4255-83f9-b093e3b99a99 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:28.204344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 729/1449] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:21:30.439379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4801e-d3cf-4e4c-b5de-0b8ebd9270e5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:30.439955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4801e-d3cf-4e4c-b5de-0b8ebd9270e5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:30.439955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4801e-d3cf-4e4c-b5de-0b8ebd9270e5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:30.441368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:30.446859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ea4b-2f38-4177-a99f-b66273b88a54 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:30.447323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ea4b-2f38-4177-a99f-b66273b88a54 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:30.447535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ea4b-2f38-4177-a99f-b66273b88a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:30.518714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bde8ea4b-2f38-4177-a99f-b66273b88a54 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:30.519936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bde8ea4b-2f38-4177-a99f-b66273b88a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:30.542122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:30.629720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:30.669779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4801e-d3cf-4e4c-b5de-0b8ebd9270e5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:30.671734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ecd4801e-d3cf-4e4c-b5de-0b8ebd9270e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:30.789183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4801e-d3cf-4e4c-b5de-0b8ebd9270e5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:30.811419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:30.872146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:30.872794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:30.873116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:30.873388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:30.875124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:30.998264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:30.999624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edc816-a5b5-4383-b422-64ddc5f8552e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:31.018870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 722/1452] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:21:33.145144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83265188-3089-4e84-b58f-dc114305faaa None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:33.145719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83265188-3089-4e84-b58f-dc114305faaa None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:33.145719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83265188-3089-4e84-b58f-dc114305faaa None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:33.147086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:33.152968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348a315-dc50-4bba-8b21-340105b39183 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:33.153422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348a315-dc50-4bba-8b21-340105b39183 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:33.153563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348a315-dc50-4bba-8b21-340105b39183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:33.218280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:33.221291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f348a315-dc50-4bba-8b21-340105b39183 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:33.223134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f348a315-dc50-4bba-8b21-340105b39183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:33.321622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:33.362633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83265188-3089-4e84-b58f-dc114305faaa None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:33.365092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-83265188-3089-4e84-b58f-dc114305faaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:33.476602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83265188-3089-4e84-b58f-dc114305faaa None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:33.527699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:33.554360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:33.555013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:33.555295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:33.555566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:33.558018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:33.672077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:33.673613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0fb43-92f5-479a-9c1e-36228307b0f8 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:33.695663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 732/1455] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:35.870950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3675f0-41ba-40f6-92f9-9450fedea211 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:35.871513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3675f0-41ba-40f6-92f9-9450fedea211 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:35.871513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3675f0-41ba-40f6-92f9-9450fedea211 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:35.872941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:35.878657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74c5f9-b8b2-4975-87e5-1521afdc8377 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:35.879093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74c5f9-b8b2-4975-87e5-1521afdc8377 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:35.879218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74c5f9-b8b2-4975-87e5-1521afdc8377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:35.944392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1b74c5f9-b8b2-4975-87e5-1521afdc8377 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:35.945679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1b74c5f9-b8b2-4975-87e5-1521afdc8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:35.974721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:36.043620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:36.084631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3675f0-41ba-40f6-92f9-9450fedea211 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:36.086218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d3675f0-41ba-40f6-92f9-9450fedea211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:36.193604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3675f0-41ba-40f6-92f9-9450fedea211 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:36.221855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:36.268162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:36.268715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:36.269032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:36.269308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:36.270977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:36.383117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:36.384622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2311e-7806-4e44-aa4e-df36dd264fcd service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:36.403242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 725/1458] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:38.560777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ace47-02e3-4b86-8afe-68b27b5637bc None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:38.561330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ace47-02e3-4b86-8afe-68b27b5637bc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:38.561449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ace47-02e3-4b86-8afe-68b27b5637bc None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:38.562770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:38.568296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85720f-d93a-43fc-a469-cfd29c6b8dc3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:38.568757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85720f-d93a-43fc-a469-cfd29c6b8dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:38.568854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85720f-d93a-43fc-a469-cfd29c6b8dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:38.636319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3f85720f-d93a-43fc-a469-cfd29c6b8dc3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:38.637584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3f85720f-d93a-43fc-a469-cfd29c6b8dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:38.645988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:38.737252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:38.776119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ace47-02e3-4b86-8afe-68b27b5637bc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:38.777734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-555ace47-02e3-4b86-8afe-68b27b5637bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:38.885914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ace47-02e3-4b86-8afe-68b27b5637bc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:38.942472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:38.960734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:38.961407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:38.961690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:38.961960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:38.963706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:39.077957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:39.079406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b74163-de3f-42ff-a988-e56fb417087b service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:39.097603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 735/1461] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:41.208073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c97f2-da17-4a74-8bef-950ab545c3aa None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:41.208627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c97f2-da17-4a74-8bef-950ab545c3aa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:41.208627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c97f2-da17-4a74-8bef-950ab545c3aa None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:41.210094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:41.215475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0a749-3590-44ff-b9da-80ae956fee52 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:41.215905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0a749-3590-44ff-b9da-80ae956fee52 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:41.216074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0a749-3590-44ff-b9da-80ae956fee52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:41.283216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2bc0a749-3590-44ff-b9da-80ae956fee52 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:41.284453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2bc0a749-3590-44ff-b9da-80ae956fee52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:41.324237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:41.383218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:41.422299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c97f2-da17-4a74-8bef-950ab545c3aa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:41.423932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d3c97f2-da17-4a74-8bef-950ab545c3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:41.545527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c97f2-da17-4a74-8bef-950ab545c3aa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:41.587022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:41.625031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:41.625681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:41.625971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:41.626322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:41.627883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:41.740293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:41.741757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7782cd-d290-4f32-9fc3-6a36dbcb7509 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:41.760778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 728/1464] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:21:43.857593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b63420-187a-4e5c-a606-926a1c1c7a81 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:43.858169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b63420-187a-4e5c-a606-926a1c1c7a81 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:43.858169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b63420-187a-4e5c-a606-926a1c1c7a81 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:43.859507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:43.865017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acf3e0-0316-4799-9c89-b69e293d18a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:43.865491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acf3e0-0316-4799-9c89-b69e293d18a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:43.865600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acf3e0-0316-4799-9c89-b69e293d18a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:43.935223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-18acf3e0-0316-4799-9c89-b69e293d18a4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:43.936472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-18acf3e0-0316-4799-9c89-b69e293d18a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:43.955339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:44.035603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:44.074375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b63420-187a-4e5c-a606-926a1c1c7a81 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:44.075918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-48b63420-187a-4e5c-a606-926a1c1c7a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:44.188763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b63420-187a-4e5c-a606-926a1c1c7a81 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:44.258939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:44.267400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:44.268045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:44.268336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:44.268620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:44.270330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:44.380804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:44.382286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859e74b-d387-4060-9a15-c46aa7890e39 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:44.400442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 738/1467] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:46.473650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d6e0f-370d-483d-bd3f-388274ee1179 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.474229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d6e0f-370d-483d-bd3f-388274ee1179 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.474229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d6e0f-370d-483d-bd3f-388274ee1179 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.475585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:46.481996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c09363-aea9-4ebb-9f54-c6e0f4c3bc26 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.482445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c09363-aea9-4ebb-9f54-c6e0f4c3bc26 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.482572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c09363-aea9-4ebb-9f54-c6e0f4c3bc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.548844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a3c09363-aea9-4ebb-9f54-c6e0f4c3bc26 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:46.550105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a3c09363-aea9-4ebb-9f54-c6e0f4c3bc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:46.603594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:46.652486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:46.690518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d6e0f-370d-483d-bd3f-388274ee1179 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:46.692124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fa3d6e0f-370d-483d-bd3f-388274ee1179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:46.796254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d6e0f-370d-483d-bd3f-388274ee1179 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:46.844421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:46.872981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:46.873635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.873919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.874194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.875848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:46.986587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:46.988020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e3a63-784a-42d1-a951-83a48a9f91ad service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:47.006729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 731/1470] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:56.650117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=22 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:57.004451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=66 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:07.088935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6383b-e899-4d93-8259-d5860aa979b9 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:07.089514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6383b-e899-4d93-8259-d5860aa979b9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:07.089514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6383b-e899-4d93-8259-d5860aa979b9 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:07.090877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 740/1471] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:07.096263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2a0b0-34fe-41d6-8f7e-246828636e92 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:07.096654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2a0b0-34fe-41d6-8f7e-246828636e92 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:07.096822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2a0b0-34fe-41d6-8f7e-246828636e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:07.162444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4aa2a0b0-34fe-41d6-8f7e-246828636e92 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:07.163658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4aa2a0b0-34fe-41d6-8f7e-246828636e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:07.380832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 732/1472] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:22:07.466519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bb6b5-b916-4dfe-9677-3388e2effef2 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:07.466810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bb6b5-b916-4dfe-9677-3388e2effef2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:07.467090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bb6b5-b916-4dfe-9677-3388e2effef2 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:07.468474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 741/1473] 10.222.0.79 () {54 vars in 966 bytes} [Wed Jul 20 19:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:22:07.473204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212107ae-35e4-4d7d-811e-70ed8144d293 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:07.473501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212107ae-35e4-4d7d-811e-70ed8144d293 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:07.473785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212107ae-35e4-4d7d-811e-70ed8144d293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:07.548336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-212107ae-35e4-4d7d-811e-70ed8144d293 None None] MFA Rules not processed for user `cd4e27030ec042bca0c7d301998e018e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:07.549559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-212107ae-35e4-4d7d-811e-70ed8144d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:07.571440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:07.628635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 733/1474] 10.222.0.79 () {58 vars in 1063 bytes} [Wed Jul 20 19:22:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:07.637117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bb6b5-b916-4dfe-9677-3388e2effef2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:07.638612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8b7bb6b5-b916-4dfe-9677-3388e2effef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:07.747044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bb6b5-b916-4dfe-9677-3388e2effef2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:07.821821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:07.822480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:07.822770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:07.823049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:07.824738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:07.878707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:07.938365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:07.939724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af151ddc-5be9-460e-a6d2-2546a2620785 service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:07.958244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 742/1475] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:10.110020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba6be6-f949-46f1-af92-760d2d107a80 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:10.110590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba6be6-f949-46f1-af92-760d2d107a80 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:10.110590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba6be6-f949-46f1-af92-760d2d107a80 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:10.112006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 734/1476] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:10.117381 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5d909-2054-4de6-82f9-9f4e90233393 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:10.118408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5d909-2054-4de6-82f9-9f4e90233393 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:10.118701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5d909-2054-4de6-82f9-9f4e90233393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:10.182889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-abd5d909-2054-4de6-82f9-9f4e90233393 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:10.184083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-abd5d909-2054-4de6-82f9-9f4e90233393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:10.279700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 743/1477] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:10.303794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba6be6-f949-46f1-af92-760d2d107a80 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:10.306033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-88ba6be6-f949-46f1-af92-760d2d107a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:10.432012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba6be6-f949-46f1-af92-760d2d107a80 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:10.485602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:10.534708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:10.535375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:10.535768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:10.535917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:10.536636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:10.538424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:22:10.538424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:22:10.538424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:22:10.540111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c83420-06ef-4617-b746-90b6fbdf5a85 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:10.559727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 735/1478] 10.222.0.79 () {56 vars in 1108 bytes} [Wed Jul 20 19:22:10 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:22:12.638735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053fb26-13f0-42a6-aeed-7f60409aaf25 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:12.639288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053fb26-13f0-42a6-aeed-7f60409aaf25 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:12.639288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053fb26-13f0-42a6-aeed-7f60409aaf25 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:12.640699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 744/1479] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:12.646006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeae4d-cbee-4512-a5de-8d9b169a33b8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:12.646439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeae4d-cbee-4512-a5de-8d9b169a33b8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:12.646564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeae4d-cbee-4512-a5de-8d9b169a33b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:12.713529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-17eeae4d-cbee-4512-a5de-8d9b169a33b8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:12.714767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-17eeae4d-cbee-4512-a5de-8d9b169a33b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:12.812774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 736/1480] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:12.856549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053fb26-13f0-42a6-aeed-7f60409aaf25 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:12.858193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b053fb26-13f0-42a6-aeed-7f60409aaf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:12.868953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:13.006462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053fb26-13f0-42a6-aeed-7f60409aaf25 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:13.088385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:13.128373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:13.129058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:13.129344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:13.129620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:13.131263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:13.220339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:13.221975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835fdb-97ec-46b2-83b5-9ffa6f16758c service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:13.240358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 745/1481] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:15.503499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cd4da-1811-4d83-a0f3-f1429cff0bf3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:15.504088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cd4da-1811-4d83-a0f3-f1429cff0bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:15.504088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cd4da-1811-4d83-a0f3-f1429cff0bf3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:15.505428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 737/1482] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:15.511329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb64eb-29a0-4e84-ad79-63d5d024cbd8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:15.511773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb64eb-29a0-4e84-ad79-63d5d024cbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:15.511907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb64eb-29a0-4e84-ad79-63d5d024cbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:15.577055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8fdb64eb-29a0-4e84-ad79-63d5d024cbd8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:15.578279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8fdb64eb-29a0-4e84-ad79-63d5d024cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:15.625358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:15.676454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 746/1483] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:15.714300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cd4da-1811-4d83-a0f3-f1429cff0bf3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:15.715873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb0cd4da-1811-4d83-a0f3-f1429cff0bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:15.740527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:15.825945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cd4da-1811-4d83-a0f3-f1429cff0bf3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:15.899602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:15.900259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:15.900544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:15.900821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:15.902519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:15.978417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:16.016466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:16.017931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e61d83-906c-4c66-b9a9-cc00ea8c0bd9 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:16.035882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 738/1484] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:18.130455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac46c6-b9ae-4822-b0ff-30e48ed59f13 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:18.131033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac46c6-b9ae-4822-b0ff-30e48ed59f13 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:18.131033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac46c6-b9ae-4822-b0ff-30e48ed59f13 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:18.132391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 747/1485] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:18.137716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937d124-af6c-40d6-aabc-6b4b72b566d5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:18.138142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937d124-af6c-40d6-aabc-6b4b72b566d5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:18.138282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937d124-af6c-40d6-aabc-6b4b72b566d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:18.202771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e937d124-af6c-40d6-aabc-6b4b72b566d5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:18.203983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e937d124-af6c-40d6-aabc-6b4b72b566d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:18.300800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 739/1486] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:18.341039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac46c6-b9ae-4822-b0ff-30e48ed59f13 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:18.342622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95ac46c6-b9ae-4822-b0ff-30e48ed59f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:18.447889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac46c6-b9ae-4822-b0ff-30e48ed59f13 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:18.517322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:18.519696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:18.520080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:18.520296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:18.520638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:18.522715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:18.637723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:18.639074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9201e3f-0057-4b32-b8a7-9b88e79a014d service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:18.657972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 748/1487] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:20.832669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6e44d-94df-4a64-820d-3a1034b136ef None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:20.833350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6e44d-94df-4a64-820d-3a1034b136ef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:20.833350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6e44d-94df-4a64-820d-3a1034b136ef None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:20.834554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 740/1488] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:20.840232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54e8a9-ab20-42a8-b22d-5cbe3f73d55a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:20.840690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54e8a9-ab20-42a8-b22d-5cbe3f73d55a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:20.840795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54e8a9-ab20-42a8-b22d-5cbe3f73d55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:20.906700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-aa54e8a9-ab20-42a8-b22d-5cbe3f73d55a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:20.907913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-aa54e8a9-ab20-42a8-b22d-5cbe3f73d55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:20.960415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:21.005796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 749/1489] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:22:21.047768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6e44d-94df-4a64-820d-3a1034b136ef None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:21.049331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-24a6e44d-94df-4a64-820d-3a1034b136ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:21.086070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:21.159588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6e44d-94df-4a64-820d-3a1034b136ef None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:21.232661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:21.233344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:21.233629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:21.233906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:21.235556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:21.316369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:21.346299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:21.347687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f76fce-bf35-4693-a2a3-1c52398518fe service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:21.365324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 741/1490] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:23.625193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3177d-3789-4055-ae0b-3d1d95e8aff9 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:23.625776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3177d-3789-4055-ae0b-3d1d95e8aff9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:23.625776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3177d-3789-4055-ae0b-3d1d95e8aff9 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:23.627159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 750/1491] 10.222.0.79 () {54 vars in 926 bytes} [Wed Jul 20 19:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:22:23.632282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631fac9-927b-493d-b1cb-1d01bf01fe49 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:23.632709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631fac9-927b-493d-b1cb-1d01bf01fe49 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:23.632818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631fac9-927b-493d-b1cb-1d01bf01fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:23.699238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7631fac9-927b-493d-b1cb-1d01bf01fe49 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:22:23.700438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7631fac9-927b-493d-b1cb-1d01bf01fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:23.799723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 742/1492] 10.222.0.79 () {58 vars in 1023 bytes} [Wed Jul 20 19:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:22:23.840677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3177d-3789-4055-ae0b-3d1d95e8aff9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:22:23.842248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b8e3177d-3789-4055-ae0b-3d1d95e8aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:23.949726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3177d-3789-4055-ae0b-3d1d95e8aff9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:22:24.023287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:22:24.023929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:22:24.024791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:22:24.025136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:22:24.026817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:22:24.036321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:24.135346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:22:24.136783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dad8a5-903e-4a2e-a598-16b07f0fe854 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:22:24.155130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 751/1493] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:33.797854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=82 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:34.152710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=50 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:51.037815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.038893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.039039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.114026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:51.115255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.212796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 743/1494] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:51.221044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.221555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.221678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.297478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:51.298624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.395925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 752/1495] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:51.401215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.402439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.414282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48e63c-94f2-4cd0-addc-7bece4479b67 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.475461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:51.509678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.510364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.510651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.510927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.511648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:51.511957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.513010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:51.513010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:23:51.513010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:23:51.513916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:51.525763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 744/1496] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:51.534354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.543963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56581b-23e3-4f08-bbc0-2415d22e0261 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.605176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:51.640014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.640677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.640998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.641278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.641965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.643753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:51.643753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:23:51.643753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:23:51.644993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:51.702241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 753/1497] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:51 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:51.706389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.707913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.741635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40482ea-fc7a-48a4-92f4-693ed618f5f9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.841268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.841944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.842229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.842508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.843188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.844198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:51.844198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:23:51.844198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:23:51.845425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:51.888686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 745/1498] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:51 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:51.892796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.894352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.956935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb8b2b-aa0b-4b50-ad3a-55729f8654f5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.044160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.057320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.058261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.058755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.059248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.060454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.061790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:52.061790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:23:52.061790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:23:52.063079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.076233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 754/1499] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:52.080273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.081849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.093543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6f0d4-4934-4a6c-b3ef-aa0c296f2570 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.121145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.195127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.195711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.195997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.196276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.197048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.198096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:52.198096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:23:52.198096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:23:52.198988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3209d6c-2873-48a7-83e8-488992735e44 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.211905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 746/1500] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:52.216126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.217719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.252527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dc6d-add7-49c7-a12b-31c4ddd8ae09 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.347402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.348430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.348716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.349027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] PATH_INFO: `/v3/projects/6666afe164804815aadc38fe85c41941/users/2915c46a45a24e71b97e33ed504e110c/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.371154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6666afe164804815aadc38fe85c41941, user_id=2915c46a45a24e71b97e33ed504e110c, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.372191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:52.372191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:23:52.372191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:23:52.374154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0c3b-4ca8-46ed-a653-d926e8ed1c4d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.403368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 755/1501] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:52 2022] PUT /identity/v3/projects/6666afe164804815aadc38fe85c41941/users/2915c46a45a24e71b97e33ed504e110c/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.417404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e09ab3-d4f5-4a68-bad8-95a77efe399f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.417703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e09ab3-d4f5-4a68-bad8-95a77efe399f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.417983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e09ab3-d4f5-4a68-bad8-95a77efe399f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.472733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d1e09ab3-d4f5-4a68-bad8-95a77efe399f None None] MFA Rules not processed for user `2915c46a45a24e71b97e33ed504e110c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.473899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1e09ab3-d4f5-4a68-bad8-95a77efe399f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.552618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 747/1502] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:52.566091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6f35-5837-4a3e-8f2b-2bc43d89b6ba None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.566384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6f35-5837-4a3e-8f2b-2bc43d89b6ba None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.566666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6f35-5837-4a3e-8f2b-2bc43d89b6ba None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.568049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 756/1503] 10.222.0.79 () {54 vars in 964 bytes} [Wed Jul 20 19:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:52.573016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389432e-91e5-4d1b-aeb4-b04876376bbf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.573314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389432e-91e5-4d1b-aeb4-b04876376bbf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.573596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389432e-91e5-4d1b-aeb4-b04876376bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.604803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.649206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d389432e-91e5-4d1b-aeb4-b04876376bbf None None] MFA Rules not processed for user `69dd61877f6e41ba86d2a9bb9537afc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.650410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d389432e-91e5-4d1b-aeb4-b04876376bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.728796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 748/1504] 10.222.0.79 () {58 vars in 1061 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:52.736557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6f35-5837-4a3e-8f2b-2bc43d89b6ba None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.738151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fd6d6f35-5837-4a3e-8f2b-2bc43d89b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.789600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.844717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6f35-5837-4a3e-8f2b-2bc43d89b6ba None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.919370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.920074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.920377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.920658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.922500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.022389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.023755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.027342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.042687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 757/1505] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:53.085278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9519-68c8-4c4e-b4ba-f99698e594f8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.085819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9519-68c8-4c4e-b4ba-f99698e594f8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.085819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9519-68c8-4c4e-b4ba-f99698e594f8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.087166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 749/1506] 10.222.0.79 () {52 vars in 829 bytes} [Wed Jul 20 19:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:53.090733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:53.091944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.191126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb114c8-566f-4503-b107-1f02945456d8 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:53.288804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:53.289849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.290136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.290416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] PATH_INFO: `/v3/users/2915c46a45a24e71b97e33ed504e110c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.305811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2915c46a45a24e71b97e33ed504e110c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.306889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:53.306889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:23:53.306889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:23:53.308151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.320627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.381465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2915c46a45a24e71b97e33ed504e110c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:53.394014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2915c46a45a24e71b97e33ed504e110c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:53.413702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2915c46a45a24e71b97e33ed504e110c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:53.414355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2915c46a45a24e71b97e33ed504e110c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:53.423468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2915c46a45a24e71b97e33ed504e110c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:53.429604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 758/1507] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:53 2022] DELETE /identity/v3/users/2915c46a45a24e71b97e33ed504e110c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:53.434097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.434388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.434668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.495796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:53.497067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.577937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.591289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 750/1508] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:53.601671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:53.602905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.705124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7eb1a9-ca30-4200-8c67-f303b6097051 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:53.882383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.903325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:53.903984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.904264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.904641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.906898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.015973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:54.017464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46839d1b-4d6f-47f5-a613-644d13e278f8 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:54.036233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 759/1509] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:54.380042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:54.381397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.393499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921cf31-ae1f-42e2-8c9c-553c3b23f921 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:54.500001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:54.500771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.501083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.501357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] PATH_INFO: `/v3/projects/6666afe164804815aadc38fe85c41941` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.511764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6666afe164804815aadc38fe85c41941)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:54.512928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:54.512928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:23:54.512928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:23:54.514192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:54.574068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6666afe164804815aadc38fe85c41941', 'request_id': 'req-53f5fab2-ec56-4966-8444-cf39bc0442e1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.580321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6666afe164804815aadc38fe85c41941', 'request_id': 'req-53f5fab2-ec56-4966-8444-cf39bc0442e1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.590293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6666afe164804815aadc38fe85c41941', 'request_id': 'req-53f5fab2-ec56-4966-8444-cf39bc0442e1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.623686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] The token cache is being invalidate because project 6666afe164804815aadc38fe85c41941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:54.623967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.624492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-53f5fab2-ec56-4966-8444-cf39bc0442e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.625892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 751/1510] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:54 2022] DELETE /identity/v3/projects/6666afe164804815aadc38fe85c41941 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:04.033928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=62 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:04.621593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:40.957412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d3ac6-a065-4447-bb06-83a65f00962a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:40.958304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d3ac6-a065-4447-bb06-83a65f00962a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:40.958304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d3ac6-a065-4447-bb06-83a65f00962a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:40.959875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:40.965128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d910-4a2b-4b0e-944d-34632f93cf94 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:40.965599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d910-4a2b-4b0e-944d-34632f93cf94 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:40.965706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d910-4a2b-4b0e-944d-34632f93cf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.048557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a326d910-4a2b-4b0e-944d-34632f93cf94 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:41.049815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a326d910-4a2b-4b0e-944d-34632f93cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:41.154697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:41.317931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d3ac6-a065-4447-bb06-83a65f00962a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:41.320313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4e7d3ac6-a065-4447-bb06-83a65f00962a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:41.441472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d3ac6-a065-4447-bb06-83a65f00962a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:41.446902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adba338-0481-457a-a69f-4203fdde1068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:41.448179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba338-0481-457a-a69f-4203fdde1068 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:41.448464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba338-0481-457a-a69f-4203fdde1068 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:41.448742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adba338-0481-457a-a69f-4203fdde1068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:41.456956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adba338-0481-457a-a69f-4203fdde1068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:41.458554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adba338-0481-457a-a69f-4203fdde1068 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:41.468669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 761/1513] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:42.301514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e07b-1f14-4810-86a4-0a178745269d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.302069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e07b-1f14-4810-86a4-0a178745269d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.302069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839e07b-1f14-4810-86a4-0a178745269d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.303467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:42.309069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1ee15-adc9-4b14-aa34-89848e9afe52 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.309486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1ee15-adc9-4b14-aa34-89848e9afe52 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.309623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1ee15-adc9-4b14-aa34-89848e9afe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.326981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e4d1ee15-adc9-4b14-aa34-89848e9afe52 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:42.328200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e4d1ee15-adc9-4b14-aa34-89848e9afe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:42.337804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:42.489211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839e07b-1f14-4810-86a4-0a178745269d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:42.504156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839e07b-1f14-4810-86a4-0a178745269d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:42.509645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeff6bd-72d8-4feb-9cb7-43010a052a10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:42.510614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeff6bd-72d8-4feb-9cb7-43010a052a10 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:42.510912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeff6bd-72d8-4feb-9cb7-43010a052a10 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:42.511197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeff6bd-72d8-4feb-9cb7-43010a052a10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:42.519328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeff6bd-72d8-4feb-9cb7-43010a052a10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:42.520990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeff6bd-72d8-4feb-9cb7-43010a052a10 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:42.531478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 754/1516] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:43.393446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953181dd-2a4d-4890-9040-a98638307dd5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:43.394024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953181dd-2a4d-4890-9040-a98638307dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:43.394024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953181dd-2a4d-4890-9040-a98638307dd5 None None] PATH_INFO: `/v3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:43.395330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 763/1517] 10.222.0.96 () {54 vars in 925 bytes} [Wed Jul 20 19:25:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:25:43.402840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05e9dc-08db-4997-b244-54496c69a8d0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:43.403235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05e9dc-08db-4997-b244-54496c69a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:43.403429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05e9dc-08db-4997-b244-54496c69a8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:43.460052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fd05e9dc-08db-4997-b244-54496c69a8d0 None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:43.461285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fd05e9dc-08db-4997-b244-54496c69a8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:43.478066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:43.535212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 755/1518] 10.222.0.96 () {58 vars in 1015 bytes} [Wed Jul 20 19:25:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:52.322031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=56 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:53.532129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=23 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:53.821906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4d35-07cb-4621-af2b-75796034e3dc None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.823140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4d35-07cb-4621-af2b-75796034e3dc None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.823478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4d35-07cb-4621-af2b-75796034e3dc None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.825751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 764/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:25:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:53.832764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8cd66-b970-4f91-821f-5aa47b32e310 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.833263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8cd66-b970-4f91-821f-5aa47b32e310 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.833586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8cd66-b970-4f91-821f-5aa47b32e310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.853658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-07b8cd66-b970-4f91-821f-5aa47b32e310 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:53.854874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-07b8cd66-b970-4f91-821f-5aa47b32e310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.864765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 756/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:54.029817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4d35-07cb-4621-af2b-75796034e3dc None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:54.048972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4d35-07cb-4621-af2b-75796034e3dc None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:54.057124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f39e6-ea0d-4d19-8a79-65ab341db713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:54.057797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f39e6-ea0d-4d19-8a79-65ab341db713 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.058077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f39e6-ea0d-4d19-8a79-65ab341db713 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.058352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f39e6-ea0d-4d19-8a79-65ab341db713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.070276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f39e6-ea0d-4d19-8a79-65ab341db713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:54.072570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f39e6-ea0d-4d19-8a79-65ab341db713 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:54.083525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 765/1521] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:54.465777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c1fd0-439f-4226-868a-c5bb70f33664 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.465992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c1fd0-439f-4226-868a-c5bb70f33664 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.466279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c1fd0-439f-4226-868a-c5bb70f33664 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.467677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 757/1522] 10.222.0.79 () {54 vars in 961 bytes} [Wed Jul 20 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:54.473127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda7f9a-e7f3-4b89-bba7-6a0a57cb167c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.473423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda7f9a-e7f3-4b89-bba7-6a0a57cb167c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.473705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda7f9a-e7f3-4b89-bba7-6a0a57cb167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.492450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2eda7f9a-e7f3-4b89-bba7-6a0a57cb167c None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:54.493675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2eda7f9a-e7f3-4b89-bba7-6a0a57cb167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:54.502898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 766/1523] 10.222.0.79 () {58 vars in 1058 bytes} [Wed Jul 20 19:25:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:54.510793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c1fd0-439f-4226-868a-c5bb70f33664 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:54.524691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c1fd0-439f-4226-868a-c5bb70f33664 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:54.530343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf32a71-329a-4d60-9db5-fccd7e32a277 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:54.530988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf32a71-329a-4d60-9db5-fccd7e32a277 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.531273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf32a71-329a-4d60-9db5-fccd7e32a277 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.531555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf32a71-329a-4d60-9db5-fccd7e32a277 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.540428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf32a71-329a-4d60-9db5-fccd7e32a277 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:54.542604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf32a71-329a-4d60-9db5-fccd7e32a277 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:54.552775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 758/1524] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:54.931815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4fb27-5a7a-49cd-b8ef-c13a4ec6516d None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.932386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4fb27-5a7a-49cd-b8ef-c13a4ec6516d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.932510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4fb27-5a7a-49cd-b8ef-c13a4ec6516d None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.933943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:54.939983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14514374-caca-4558-9e31-684fa7fe2449 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.940275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14514374-caca-4558-9e31-684fa7fe2449 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.940561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14514374-caca-4558-9e31-684fa7fe2449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.959417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-14514374-caca-4558-9e31-684fa7fe2449 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:54.960619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-14514374-caca-4558-9e31-684fa7fe2449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:54.969525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:55.126311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4fb27-5a7a-49cd-b8ef-c13a4ec6516d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:55.141322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4fb27-5a7a-49cd-b8ef-c13a4ec6516d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:55.146764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc053dde-d469-479d-9a10-1468789be921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.147421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053dde-d469-479d-9a10-1468789be921 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.147702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053dde-d469-479d-9a10-1468789be921 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.147975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053dde-d469-479d-9a10-1468789be921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.158436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc053dde-d469-479d-9a10-1468789be921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.160215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc053dde-d469-479d-9a10-1468789be921 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.174368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 768/1527] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:59.169019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952e7e-62e2-4c4c-8ce2-bb2cf2aeabd0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.169547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952e7e-62e2-4c4c-8ce2-bb2cf2aeabd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.169547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952e7e-62e2-4c4c-8ce2-bb2cf2aeabd0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.170961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:59.176147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c003d-f02a-4404-8999-6b73916af8b6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.176554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c003d-f02a-4404-8999-6b73916af8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.176713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c003d-f02a-4404-8999-6b73916af8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.199925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f45c003d-f02a-4404-8999-6b73916af8b6 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:59.201190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f45c003d-f02a-4404-8999-6b73916af8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:59.209382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:59.284569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8952e7e-62e2-4c4c-8ce2-bb2cf2aeabd0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:59.286255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8952e7e-62e2-4c4c-8ce2-bb2cf2aeabd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:59.364707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8952e7e-62e2-4c4c-8ce2-bb2cf2aeabd0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:59.368276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e90761-9b3b-4452-a461-060ffe986084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:59.368938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e90761-9b3b-4452-a461-060ffe986084 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:59.369784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e90761-9b3b-4452-a461-060ffe986084 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:59.370054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e90761-9b3b-4452-a461-060ffe986084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:59.379184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e90761-9b3b-4452-a461-060ffe986084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:59.380604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e90761-9b3b-4452-a461-060ffe986084 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:59.390152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 761/1530] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:01.462012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6a70-8c83-4732-8913-806eafc8bb3a None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:01.462603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6a70-8c83-4732-8913-806eafc8bb3a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:01.463234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6a70-8c83-4732-8913-806eafc8bb3a None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:01.464744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:26:01.470097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed276-4c9c-4792-8783-a21a2a91c215 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:01.470544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed276-4c9c-4792-8783-a21a2a91c215 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:01.470676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ed276-4c9c-4792-8783-a21a2a91c215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:01.488895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0a1ed276-4c9c-4792-8783-a21a2a91c215 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:01.490096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0a1ed276-4c9c-4792-8783-a21a2a91c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:01.498253 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:01.534892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6a70-8c83-4732-8913-806eafc8bb3a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:01.546575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6a70-8c83-4732-8913-806eafc8bb3a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:01.550030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3b5ec-553a-4a86-9045-08c26d650c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:01.550663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b5ec-553a-4a86-9045-08c26d650c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:01.550946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b5ec-553a-4a86-9045-08c26d650c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:01.551222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3b5ec-553a-4a86-9045-08c26d650c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:01.560117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3b5ec-553a-4a86-9045-08c26d650c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:01.561564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3b5ec-553a-4a86-9045-08c26d650c2f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:01.570774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 771/1533] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:05.160498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f198b3-b98d-4667-b8f3-9ce61d600a0b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:05.161015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f198b3-b98d-4667-b8f3-9ce61d600a0b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:05.161113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f198b3-b98d-4667-b8f3-9ce61d600a0b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:05.162502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:26:05.168668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc230069-bba1-40bf-a210-4f2a8a876036 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:05.169215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc230069-bba1-40bf-a210-4f2a8a876036 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:05.169726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc230069-bba1-40bf-a210-4f2a8a876036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:05.187520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fc230069-bba1-40bf-a210-4f2a8a876036 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:05.188737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fc230069-bba1-40bf-a210-4f2a8a876036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:05.196616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:05.509290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f198b3-b98d-4667-b8f3-9ce61d600a0b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:05.511589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9f198b3-b98d-4667-b8f3-9ce61d600a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:05.593493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f198b3-b98d-4667-b8f3-9ce61d600a0b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:05.597325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69227c-864f-4419-a6f5-c233443f30e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:05.597970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69227c-864f-4419-a6f5-c233443f30e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:05.598254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69227c-864f-4419-a6f5-c233443f30e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:05.598532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69227c-864f-4419-a6f5-c233443f30e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:05.609998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69227c-864f-4419-a6f5-c233443f30e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:05.611427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69227c-864f-4419-a6f5-c233443f30e7 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:05.622649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 764/1536] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:07.698657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fc420-9518-448e-bba4-6dd4f5778ac6 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:07.699193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fc420-9518-448e-bba4-6dd4f5778ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:07.699193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fc420-9518-448e-bba4-6dd4f5778ac6 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:07.700932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:26:07.705993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f35e5-86d4-4881-b7ef-35f6d4974767 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:07.706386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f35e5-86d4-4881-b7ef-35f6d4974767 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:07.706553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f35e5-86d4-4881-b7ef-35f6d4974767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:07.724366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b43f35e5-86d4-4881-b7ef-35f6d4974767 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:07.725601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b43f35e5-86d4-4881-b7ef-35f6d4974767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:07.733538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:08.045239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fc420-9518-448e-bba4-6dd4f5778ac6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:08.047059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9c0fc420-9518-448e-bba4-6dd4f5778ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:08.061456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fc420-9518-448e-bba4-6dd4f5778ac6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:08.065231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5603a-cb27-4d74-975f-a1d5689a94e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:08.066047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5603a-cb27-4d74-975f-a1d5689a94e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:08.066337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5603a-cb27-4d74-975f-a1d5689a94e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:08.066608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5603a-cb27-4d74-975f-a1d5689a94e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:08.076523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5603a-cb27-4d74-975f-a1d5689a94e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:08.078068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5603a-cb27-4d74-975f-a1d5689a94e6 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:08.089286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 774/1539] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:17.719626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:18.084041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=33 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:38.101851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1054-22ce-481c-bcbd-ca052d47df93 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.103149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1054-22ce-481c-bcbd-ca052d47df93 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.103149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1054-22ce-481c-bcbd-ca052d47df93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.135701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-245b1054-22ce-481c-bcbd-ca052d47df93 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.137089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-245b1054-22ce-481c-bcbd-ca052d47df93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.152204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 766/1540] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.155002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2961e8-19ad-420d-8771-b2584afba187 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.155915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2961e8-19ad-420d-8771-b2584afba187 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.156415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2961e8-19ad-420d-8771-b2584afba187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.161405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.161700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.161985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.186782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.188220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.188501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9a2961e8-19ad-420d-8771-b2584afba187 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.189744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9a2961e8-19ad-420d-8771-b2584afba187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.204955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 767/1541] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:38.207957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 775/1542] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.209390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.218221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.218581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.218962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.222699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec41ac7-666e-4bc7-ab48-f5ba4d05fe7e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.243681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.244392 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.244683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.244996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.245760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.246072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.247972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.266187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.267442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.281580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 768/1543] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.289569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.291311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 776/1544] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.296212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.309255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e66103-3464-4c94-8405-69b69085e57d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.311964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daef5af-e924-40f6-9f60-78931c7f13ac None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.322711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.322949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.323402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.323529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.323681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.323847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.324027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.324436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.324736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.325542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.326022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.326165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:27:38.326165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:27:38.326165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:27:38.327286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a807996-288f-470f-934f-6aed17f45751 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.330951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:27:38.330951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:27:38.330951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:27:38.332177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1afd45-2a80-4f21-bbfa-ceb540281060 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.361711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 777/1545] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.363434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275845-2cb9-44c5-b545-3ed831c37c78 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.363901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275845-2cb9-44c5-b545-3ed831c37c78 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.364351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275845-2cb9-44c5-b545-3ed831c37c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.396650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-03275845-2cb9-44c5-b545-3ed831c37c78 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.398665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-03275845-2cb9-44c5-b545-3ed831c37c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.403735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 769/1546] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:27:38.405403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b37298-8c99-47d2-a737-052a64d7adfe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.405902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b37298-8c99-47d2-a737-052a64d7adfe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.406380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b37298-8c99-47d2-a737-052a64d7adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.412281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 778/1547] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.415338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.416050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.416745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.426442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-85b37298-8c99-47d2-a737-052a64d7adfe None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.427679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-85b37298-8c99-47d2-a737-052a64d7adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.447633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.448985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.509127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 770/1548] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.516773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0726cb-0ce2-43ae-8a45-54cae2d53844 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.516773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0726cb-0ce2-43ae-8a45-54cae2d53844 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.516773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0726cb-0ce2-43ae-8a45-54cae2d53844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.526390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 779/1549] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.527613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.536950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4e0726cb-0ce2-43ae-8a45-54cae2d53844 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.538170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4e0726cb-0ce2-43ae-8a45-54cae2d53844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.543405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83bfef-b4da-4330-9054-d1fd84737650 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.547792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 771/1550] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.548635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c69cf5-e953-4963-8846-ab9356c560d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.549011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.549295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.549487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c69cf5-e953-4963-8846-ab9356c560d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.549650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.549801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c69cf5-e953-4963-8846-ab9356c560d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.550056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c69cf5-e953-4963-8846-ab9356c560d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.550816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c69cf5-e953-4963-8846-ab9356c560d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.553576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c69cf5-e953-4963-8846-ab9356c560d8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.570586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.572540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.584792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 772/1551] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.586330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.599909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 780/1552] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:38.600139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfc876-f101-4661-a71f-bb1c04424471 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.600247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e7579-a5f7-44f1-92f2-7cea280db30e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.600247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e7579-a5f7-44f1-92f2-7cea280db30e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.600247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e7579-a5f7-44f1-92f2-7cea280db30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.622996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-249e7579-a5f7-44f1-92f2-7cea280db30e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.624212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-249e7579-a5f7-44f1-92f2-7cea280db30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.675219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f139f74-0c65-4bf7-b48d-7b37a0337849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.675907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f139f74-0c65-4bf7-b48d-7b37a0337849 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.676192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f139f74-0c65-4bf7-b48d-7b37a0337849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.676500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f139f74-0c65-4bf7-b48d-7b37a0337849 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.677232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f139f74-0c65-4bf7-b48d-7b37a0337849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.679189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f139f74-0c65-4bf7-b48d-7b37a0337849 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.709335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 781/1553] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.709736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 773/1554] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:38.711765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcecc8-2342-4ee9-b243-d56017a57418 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.712127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcecc8-2342-4ee9-b243-d56017a57418 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.712415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcecc8-2342-4ee9-b243-d56017a57418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.712746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee4b2-f3de-49ba-b896-1155fc677bda None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.713922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee4b2-f3de-49ba-b896-1155fc677bda None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.714410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee4b2-f3de-49ba-b896-1155fc677bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.735735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-13fcecc8-2342-4ee9-b243-d56017a57418 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.737012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-13fcecc8-2342-4ee9-b243-d56017a57418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.740559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7bfee4b2-f3de-49ba-b896-1155fc677bda None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.741780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7bfee4b2-f3de-49ba-b896-1155fc677bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.746264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 774/1555] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.750648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d738b-b913-44ee-8acf-3e6ee1478c54 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.751090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d738b-b913-44ee-8acf-3e6ee1478c54 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.751500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d738b-b913-44ee-8acf-3e6ee1478c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.756616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 782/1556] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.756616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.756616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.756616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.775418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b63d738b-b913-44ee-8acf-3e6ee1478c54 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.776642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b63d738b-b913-44ee-8acf-3e6ee1478c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.780866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.782094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.785505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 775/1557] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.786758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.787045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.787322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.793483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 783/1558] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.794288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.801700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:38.807601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dc6a0-8769-43cd-934b-f836e8b03ae3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.809406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.811334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.812341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.813053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.813343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.813623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.814313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.815332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:27:38.815332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:27:38.815332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:27:38.816541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.819333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 776/1559] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.820124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.830852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228178f-d92e-4498-bc22-1438be1e4715 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.837086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.837787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.838072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.838346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.839210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.839456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.841759 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90546069-0a8e-4c15-bfd3-5c0df4f250ef admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.842792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 784/1560] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:38.843636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.856852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 777/1561] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.857812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607daf29-4289-424a-887b-f5dfe70df248 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.859211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.860861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.861395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.863049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.863799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.864098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.864391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.865149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.868383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.879895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 785/1562] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:38.881199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.881313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:38.882791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:38.888523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:38.891839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 778/1563] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:38.892748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.893597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425ead7-67c2-40a9-92fe-c37060facfc2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.901380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.902257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.902645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.902931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.903712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.904024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.904308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2d437-b068-4cb5-8852-74da42d3f552 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.905805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.910576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.911255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.911538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.911819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.912555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.912862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.915309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.931492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 786/1564] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.931492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.938903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672a684-2bfc-43cc-8a33-e99a63884824 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.941233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 779/1565] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.942351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.945697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.947035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.947650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.948250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.949577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.949955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.951670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.956077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fadef3-391f-48d2-80f3-748ccd6908a2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.963157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.963833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.964123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.964400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.965169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:38.965479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.967413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.967918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 787/1566] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.969323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:38.980811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537676f-681e-4327-a791-fa7552ef369f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:38.986859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:38.987533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:38.987828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:38.988105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:38.989409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:38.997026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 780/1567] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:38.997250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:38.997350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.003145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96445b2-aeab-49a1-8ae6-44e6adf3c087 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.008286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 788/1568] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.008489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.008949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.009272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.009498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.009659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.010189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.012152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.022819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dbd7b-f10b-4775-b0b1-d08b3d54b3c8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.028487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.029198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.029491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.029771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.030632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.032405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.045206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 781/1569] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:38 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.052509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.052689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 789/1570] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.052689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.061110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a2354-d4c7-4e56-be56-de8c04eeb055 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.119651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a7683-ab80-4f5a-a83a-57c7dfea5262 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.126477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.127150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.127433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.127711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.128612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:39.128952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.130632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.134878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.135645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.136009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.136359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.137189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.139309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.144594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 782/1571] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:39.145450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.162297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4211178-0c88-4e4b-acdc-2d4b7104ba5f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.168830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.169899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.170356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.170819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.172000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.174107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.176680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 790/1572] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.177545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.204617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 783/1573] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.205735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.257312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e2562-112b-4aea-9a0e-0d4d5b459607 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.278497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad88a6c-86b2-4431-8910-f528d1c2fa0d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.285128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.286225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.286690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.287123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.288038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.290224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.300054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:39.311078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:39.318993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 784/1574] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.320812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.326956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.327712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.328071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.328454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.329271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.331187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.360914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 791/1575] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/projects => generated 393 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.360914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.391022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef88bb-2cbf-461a-bea7-2fbfa7caea1c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.424407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd23a74-0cd6-41cc-a417-b91672042b25 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.429920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.430706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.430991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.432085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.432085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.433970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.456370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 792/1576] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.457395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.468014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.468691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.469010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.469286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.470014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.470150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20971ba-9d75-4391-a7bf-c60bf2c352d0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.471623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.476690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.477738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.478027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.478306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] PATH_INFO: `/v3/projects/509e9c4cf31b4e9ea9d42a7bc4b99ac3/users/b853c55a3b6f4f2cb7dc6dc68cb64826/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.482836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 785/1577] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.486915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.496048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509e9c4cf31b4e9ea9d42a7bc4b99ac3, user_id=b853c55a3b6f4f2cb7dc6dc68cb64826, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.497619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6979a-55df-4087-a55b-5a2ee9f09884 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.498766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.503977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.504638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.504993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.505266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.506025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.508717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.515570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 793/1578] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:39 2022] PUT /identity/v3/projects/509e9c4cf31b4e9ea9d42a7bc4b99ac3/users/b853c55a3b6f4f2cb7dc6dc68cb64826/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:39.519097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.543589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 786/1579] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.543589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.599185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6faa4-8ef6-4a37-9f68-26cb6ee5a2d4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.627002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1bf58-2783-4f29-a3ee-01f71574c823 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.632039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.633745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.633919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.634260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.634903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.637527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.652513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:39.656089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:39.666933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.667612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.667895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.668171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.668860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.669490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 787/1580] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.670631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.670839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.681622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b2eb-9771-455c-8def-3f52e5acaa16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.686703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.687229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.687514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.687792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.688476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.690431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.699363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 794/1581] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.699363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.709111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76548-73ea-4ac7-bd68-35e6a37207a0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.716172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 788/1582] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.716172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.716398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.716398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.716398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.716398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.716763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.718370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.724051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ab8a-6f58-49cf-8151-88ea51edcac3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.728849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc0ca5-2415-4a20-832f-a66e85e30c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.729535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0ca5-2415-4a20-832f-a66e85e30c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.729819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0ca5-2415-4a20-832f-a66e85e30c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.730147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0ca5-2415-4a20-832f-a66e85e30c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.731055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc0ca5-2415-4a20-832f-a66e85e30c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.732783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc0ca5-2415-4a20-832f-a66e85e30c7a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.739804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 795/1583] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.740626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.746867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 789/1584] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.748173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.748466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.748746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.754418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20e7c0-0806-4e4e-9710-02b2f1513711 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.759723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.760626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.760955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.761238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] PATH_INFO: `/v3/projects/01498bd6bd0848519576ff941722e05f/users/47cbc04374bb4c35af479290e62c8109/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.776569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01498bd6bd0848519576ff941722e05f, user_id=47cbc04374bb4c35af479290e62c8109, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.776911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] MFA Rules not processed for user `b853c55a3b6f4f2cb7dc6dc68cb64826`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:39.778154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:39.779273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.793374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 796/1585] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:39 2022] PUT /identity/v3/projects/01498bd6bd0848519576ff941722e05f/users/47cbc04374bb4c35af479290e62c8109/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:39.794636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.857710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 790/1586] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:39.858566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.868783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81045db-7122-4454-bffd-45ee6efa170d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.873058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb93dd4-ad02-463f-acd5-ea62eae63bad None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.875356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.877560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.878124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.878124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.879002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.879598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.879829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.880188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.880542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.881764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.882057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.883605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.898980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 791/1587] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.903338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.912265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 797/1588] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/users => generated 303 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:39.912474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.916693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26c92-5196-4a29-b69e-8ed861883b43 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.921547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.922441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.922832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.923136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3710e-9710-4446-aa94-f4d0f338e893 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.923254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.924045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.925767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.928114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.928785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.929103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.929387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.930858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.932493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.941083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 792/1589] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.941948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.948221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 798/1590] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.949915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:39.952569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e1095-8ba4-4348-8ce2-efb4f98aa87b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.958560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b9628-dff1-4376-ad2e-33802f49a79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.959238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b9628-dff1-4376-ad2e-33802f49a79d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.959525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b9628-dff1-4376-ad2e-33802f49a79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.959853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b9628-dff1-4376-ad2e-33802f49a79d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.960345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61f54d-b9d3-4503-acc9-4084da2c6b12 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:39.960540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b9628-dff1-4376-ad2e-33802f49a79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.962232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b9628-dff1-4376-ad2e-33802f49a79d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.966321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea6164-f5b7-4430-b951-388e04bf4cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:39.967022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea6164-f5b7-4430-b951-388e04bf4cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.967310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea6164-f5b7-4430-b951-388e04bf4cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.967589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea6164-f5b7-4430-b951-388e04bf4cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.968331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea6164-f5b7-4430-b951-388e04bf4cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:39.969963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea6164-f5b7-4430-b951-388e04bf4cdf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:39.973857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 793/1591] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.975126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.975413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.975692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.984514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 799/1592] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:39.986039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03016378-6adf-4ef7-ab38-e19bc33e2dc5 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:39.986409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03016378-6adf-4ef7-ab38-e19bc33e2dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:39.986752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03016378-6adf-4ef7-ab38-e19bc33e2dc5 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:39.988150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 800/1593] 10.222.0.79 () {54 vars in 962 bytes} [Wed Jul 20 19:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:27:39.989747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03016378-6adf-4ef7-ab38-e19bc33e2dc5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.003743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03016378-6adf-4ef7-ab38-e19bc33e2dc5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.006300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] MFA Rules not processed for user `47cbc04374bb4c35af479290e62c8109`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.007771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.008736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.009432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.009757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.010036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.010777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.012591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.023824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 801/1594] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:40.024741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.035316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93339-9544-427a-a20b-2268fcaab445 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.041016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.041695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.041976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.042245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.042976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.044570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.055460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 802/1595] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:40.056378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.066572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacfaf6-f279-42f3-ac17-075ad9260a26 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.071459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.072131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.072404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.072676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.073434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.075071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.080110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 794/1596] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.083957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.086099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 803/1597] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:40.087263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.095188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69141-b9e1-4ad0-a3c0-47d52628f3ab None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.097385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c36032-fb94-4e42-95c6-cf789168e681 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.099999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.100711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.101061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.101997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.102747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.102928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1b9b4-345e-4d7a-a5ab-15a59e01536e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.103701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1b9b4-345e-4d7a-a5ab-15a59e01536e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.104298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1b9b4-345e-4d7a-a5ab-15a59e01536e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.104298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1b9b4-345e-4d7a-a5ab-15a59e01536e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.105170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.105478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1b9b4-345e-4d7a-a5ab-15a59e01536e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.111781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1b9b4-345e-4d7a-a5ab-15a59e01536e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.116201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 795/1598] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:40.121443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.128477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 804/1599] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:40.128477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.128477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.128477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.128965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82aa49-8575-4ca6-b9e8-e04da1563a07 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.133903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.134803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.135091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.135378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] PATH_INFO: `/v3/projects/183d6a3ee79e4fedb83591ce60694283/users/a271ea90c6884817991aaa9e8be36cab/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.138200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:40.157808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=183d6a3ee79e4fedb83591ce60694283, user_id=a271ea90c6884817991aaa9e8be36cab, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.158629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:27:40.158629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:27:40.158629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:27:40.160592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.164844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:40.174316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.175558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.176275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 796/1600] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:40 2022] PUT /identity/v3/projects/183d6a3ee79e4fedb83591ce60694283/users/a271ea90c6884817991aaa9e8be36cab/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:40.177111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.245786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d5acd-1bae-4c78-ba83-5e6fab0804c5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.248159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 805/1601] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.249033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.251358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.252250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.252537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.252816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] PATH_INFO: `/v3/projects/a2cb0c127060486b89ae4e2eb60bdbc6/users/c225586210fd4563bc61f2587961271b/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.259474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9ee50-5492-4484-a068-52eddc4b8209 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.264988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.264988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.265274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.265383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.266056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.267681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.268580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cb0c127060486b89ae4e2eb60bdbc6, user_id=c225586210fd4563bc61f2587961271b, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.271322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.279433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 806/1602] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:40.280337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.286425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 797/1603] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:40 2022] PUT /identity/v3/projects/a2cb0c127060486b89ae4e2eb60bdbc6/users/c225586210fd4563bc61f2587961271b/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:40.287215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.293931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a61696-ebc3-42bf-b509-fd70ee735de5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.360695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.361958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.362280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.362559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.363257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.365199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.369467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece51ab-2570-4a9d-9039-bceb4cb4ecf2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.374239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.375178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.375491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.375737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] PATH_INFO: `/v3/projects/02f994a0231b4176b8181d13d58ee9ec/users/3f7758d298f642138b71aa422d0344da/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.393285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f994a0231b4176b8181d13d58ee9ec, user_id=3f7758d298f642138b71aa422d0344da, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.396061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.402537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 807/1604] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:40.403767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.414688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 798/1605] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:40 2022] PUT /identity/v3/projects/02f994a0231b4176b8181d13d58ee9ec/users/3f7758d298f642138b71aa422d0344da/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:40.415915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.479113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:40.480567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9831b2-c12d-476d-b3af-2f1a4682a496 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.496590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad554be-112c-43a6-a27d-bf33736c0920 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.498697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:40.504977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99352c7-9ea2-492c-99cf-1fa65aa05d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.506770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99352c7-9ea2-492c-99cf-1fa65aa05d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.507068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99352c7-9ea2-492c-99cf-1fa65aa05d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.507351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99352c7-9ea2-492c-99cf-1fa65aa05d15 admin admin] PATH_INFO: `/v3/projects/9c025843a2de49d48c39b40f2261d3bf/users/347106fdd7c94624ab9d40b51da78623/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.523109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99352c7-9ea2-492c-99cf-1fa65aa05d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c025843a2de49d48c39b40f2261d3bf, user_id=347106fdd7c94624ab9d40b51da78623, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.526302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99352c7-9ea2-492c-99cf-1fa65aa05d15 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.541547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 799/1606] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:40 2022] PUT /identity/v3/projects/9c025843a2de49d48c39b40f2261d3bf/users/347106fdd7c94624ab9d40b51da78623/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:40.543099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.543389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.543666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.544600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.545541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.545813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.546091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] PATH_INFO: `/v3/projects/ca9f11498f9d4245910281ac3bd55470/users/c18c8d4034ce40dba9f9f48e96a19814/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.560851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.562097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.564612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9f11498f9d4245910281ac3bd55470, user_id=c18c8d4034ce40dba9f9f48e96a19814, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.567666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.582828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 808/1607] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:40 2022] PUT /identity/v3/projects/ca9f11498f9d4245910281ac3bd55470/users/c18c8d4034ce40dba9f9f48e96a19814/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:40.583801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.637263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 800/1608] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.638163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.649135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b040287-1b4f-4978-8f54-b64fee05b0c1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.670477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5da60-1b64-4995-a104-f19ea4e5b55a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.675902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.676554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.676841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.677149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.707969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735337ce-eaf0-4b1d-8d3b-674913f6a021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.708866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735337ce-eaf0-4b1d-8d3b-674913f6a021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.709171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735337ce-eaf0-4b1d-8d3b-674913f6a021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.709446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735337ce-eaf0-4b1d-8d3b-674913f6a021 admin admin] PATH_INFO: `/v3/projects/414eac1ab7eb41f284cb191a2388a398/users/9380df8196df4175b8d5e87f61a70e55/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.730352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735337ce-eaf0-4b1d-8d3b-674913f6a021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414eac1ab7eb41f284cb191a2388a398, user_id=9380df8196df4175b8d5e87f61a70e55, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.733095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735337ce-eaf0-4b1d-8d3b-674913f6a021 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.743515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.745166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.747396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 801/1609] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:40 2022] PUT /identity/v3/projects/414eac1ab7eb41f284cb191a2388a398/users/9380df8196df4175b8d5e87f61a70e55/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:40.748982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a720e-b2ec-4f69-bbea-1fa67d94bf03 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.749726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a720e-b2ec-4f69-bbea-1fa67d94bf03 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.749726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a720e-b2ec-4f69-bbea-1fa67d94bf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.754688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 809/1610] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:40.755478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.766381 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94c1ab-1519-4609-b806-3a9fa10230da service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.786578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-203a720e-b2ec-4f69-bbea-1fa67d94bf03 None None] MFA Rules not processed for user `c225586210fd4563bc61f2587961271b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.788432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-203a720e-b2ec-4f69-bbea-1fa67d94bf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.812592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:40.839458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:40.845468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ca17e-3e7d-40a8-9eed-5c5f9bf96be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.846259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ca17e-3e7d-40a8-9eed-5c5f9bf96be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.846583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ca17e-3e7d-40a8-9eed-5c5f9bf96be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.846937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ca17e-3e7d-40a8-9eed-5c5f9bf96be8 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.847778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ca17e-3e7d-40a8-9eed-5c5f9bf96be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.850003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ca17e-3e7d-40a8-9eed-5c5f9bf96be8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:40.865950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 802/1611] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.867566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2e92-9f6c-4bca-8ad2-d4e9ea92ca7c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.867957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2e92-9f6c-4bca-8ad2-d4e9ea92ca7c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.868294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2e92-9f6c-4bca-8ad2-d4e9ea92ca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.878109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 810/1612] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:40.879824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.880392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.880859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.887987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3d0f2e92-9f6c-4bca-8ad2-d4e9ea92ca7c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.889810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d0f2e92-9f6c-4bca-8ad2-d4e9ea92ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.900500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 803/1613] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.901701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f898f9-5e7d-4b2c-85f5-f3ff7f00a3f1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.902042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f898f9-5e7d-4b2c-85f5-f3ff7f00a3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.902397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f898f9-5e7d-4b2c-85f5-f3ff7f00a3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.905804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.906280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.916114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 811/1614] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.920056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.931767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c7f898f9-5e7d-4b2c-85f5-f3ff7f00a3f1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.933151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c7f898f9-5e7d-4b2c-85f5-f3ff7f00a3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.946394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 804/1615] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.946394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.946394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.946394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.964683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:40.966029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:40.976764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 805/1616] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:40.977613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:40.988306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135578b4-aa47-48d4-9d8c-c720cf1d9861 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:40.993981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:40.994809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:40.995143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:40.995486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:40.996260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:40.998318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.003273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbc71e-e766-4bb7-95ec-c46b658baa7f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.011477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.012147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.012434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.012712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.023751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.025375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.037473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 812/1617] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:41.037810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 806/1618] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:40 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:41.039102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.039350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.041064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:41.055227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a1884-000b-4f1c-82bb-de867155623a service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.118650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b0c68-18ac-4ae0-bd6e-cfbb38b72351 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.129342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.130359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.130721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.131069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.131892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.133743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.138176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.139177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.139693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.140427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.146742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 807/1619] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:41.147418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.155121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.157176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.159937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccae4c-5d76-4f13-a219-b3c4cbdfb049 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.165068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.165910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.166633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.166986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.174578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 813/1620] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:41.176158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.178639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.181166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.186685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48e2bd-9d46-43c7-a4cf-3411c9ba86b7 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.191219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.192156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.192684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.193216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.198898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 808/1621] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:41.201185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.205582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.207505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.221112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 814/1622] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:41.222612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.223842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f3546-c761-4ff9-9cfe-cad956c76bd2 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.233863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.234582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.234874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.235148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.237890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c190-01c6-4b61-be4c-3a089ee1c8bc service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.245145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.246372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.246921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.247554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.248604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.251684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.259624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.262649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.295716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 809/1623] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:41.296349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.303508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 815/1624] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:41 2022] POST /identity/v3/users => generated 289 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:41.307394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.314357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a59eb-3ba3-4300-8dad-bf1d10b76c85 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.321684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9456b-d662-4a1d-b51b-801c102aa90e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.327116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.328047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.328501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.329062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.330065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093ec0b-14a6-430e-86bc-aa5b9e748eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.330336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.330979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ec0b-14a6-430e-86bc-aa5b9e748eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.331265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ec0b-14a6-430e-86bc-aa5b9e748eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.331538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ec0b-14a6-430e-86bc-aa5b9e748eee admin admin] PATH_INFO: `/v3/projects/bff9680748f64a15a9eba7110b8ea565/users/87e05e88b9c44eeb9f7c4f2169c1f50b/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.333019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.347259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 816/1625] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:27:41.352073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.365074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ec0b-14a6-430e-86bc-aa5b9e748eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff9680748f64a15a9eba7110b8ea565, user_id=87e05e88b9c44eeb9f7c4f2169c1f50b, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.371301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ec0b-14a6-430e-86bc-aa5b9e748eee admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.378037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451695e-cdd6-40df-a62f-20e4de65c6ea admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.394404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983f937-25bf-4ca2-bb88-115b8429f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.396924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f937-25bf-4ca2-bb88-115b8429f17a admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.398251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f937-25bf-4ca2-bb88-115b8429f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.399121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f937-25bf-4ca2-bb88-115b8429f17a admin admin] PATH_INFO: `/v3/projects/7b58c19fc5d440b0a90ed88e91c0acd3/users/0b386c19ab754ffc8bfa314304058eed/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.406765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 810/1626] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:41 2022] PUT /identity/v3/projects/bff9680748f64a15a9eba7110b8ea565/users/87e05e88b9c44eeb9f7c4f2169c1f50b/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:41.407588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f50f37-8182-47dd-8c23-0335e8a6921f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.408982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f50f37-8182-47dd-8c23-0335e8a6921f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.409296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f50f37-8182-47dd-8c23-0335e8a6921f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.411350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 811/1627] 10.222.0.79 () {54 vars in 962 bytes} [Wed Jul 20 19:27:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:27:41.414248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f50f37-8182-47dd-8c23-0335e8a6921f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.434184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f937-25bf-4ca2-bb88-115b8429f17a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b58c19fc5d440b0a90ed88e91c0acd3, user_id=0b386c19ab754ffc8bfa314304058eed, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.437728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f937-25bf-4ca2-bb88-115b8429f17a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.450355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:41.457807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 817/1628] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:41 2022] PUT /identity/v3/projects/7b58c19fc5d440b0a90ed88e91c0acd3/users/0b386c19ab754ffc8bfa314304058eed/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:41.459601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.460119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.460468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.485153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:41.487176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:41.495555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:41.521740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f50f37-8182-47dd-8c23-0335e8a6921f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.597309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 818/1629] 10.222.0.79 () {58 vars in 1059 bytes} [Wed Jul 20 19:27:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:41.604665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.616820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e22f-5549-4a69-896f-e76080354c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.617991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e22f-5549-4a69-896f-e76080354c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.618866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e22f-5549-4a69-896f-e76080354c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.619237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e22f-5549-4a69-896f-e76080354c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.620408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402e22f-5549-4a69-896f-e76080354c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.622554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402e22f-5549-4a69-896f-e76080354c94 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.636768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 812/1630] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:41.638390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269911d-f761-420f-964c-0f3776fda919 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.638799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269911d-f761-420f-964c-0f3776fda919 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.639224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269911d-f761-420f-964c-0f3776fda919 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.641072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 813/1631] 10.222.0.79 () {54 vars in 912 bytes} [Wed Jul 20 19:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:27:41.642115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269911d-f761-420f-964c-0f3776fda919 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.662972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269911d-f761-420f-964c-0f3776fda919 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.670373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafcd9a-b1fc-45a3-adae-43ef7ae05f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.672186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafcd9a-b1fc-45a3-adae-43ef7ae05f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.672596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafcd9a-b1fc-45a3-adae-43ef7ae05f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.673045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafcd9a-b1fc-45a3-adae-43ef7ae05f1a admin admin] PATH_INFO: `/v3/projects/bff9680748f64a15a9eba7110b8ea565/users/87e05e88b9c44eeb9f7c4f2169c1f50b/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.676158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafcd9a-b1fc-45a3-adae-43ef7ae05f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff9680748f64a15a9eba7110b8ea565, user_id=87e05e88b9c44eeb9f7c4f2169c1f50b, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.680718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafcd9a-b1fc-45a3-adae-43ef7ae05f1a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.697033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 814/1632] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:41 2022] PUT /identity/v3/projects/bff9680748f64a15a9eba7110b8ea565/users/87e05e88b9c44eeb9f7c4f2169c1f50b/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:41.698524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.698825 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.699159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.720005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d38ed-404a-4d9a-a967-e044a5bea3b1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.735110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:41.736764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:41.806461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d51d3-985f-43ca-bfde-5eb9743a0e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.807633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d51d3-985f-43ca-bfde-5eb9743a0e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.807968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d51d3-985f-43ca-bfde-5eb9743a0e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.808440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d51d3-985f-43ca-bfde-5eb9743a0e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.809350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d51d3-985f-43ca-bfde-5eb9743a0e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.811711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d51d3-985f-43ca-bfde-5eb9743a0e7d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.828824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 819/1633] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:41.831743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9edb16-2d62-4d08-ab2b-874c3d4518fe None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.832162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9edb16-2d62-4d08-ab2b-874c3d4518fe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.832795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9edb16-2d62-4d08-ab2b-874c3d4518fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.880638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0c9edb16-2d62-4d08-ab2b-874c3d4518fe None None] MFA Rules not processed for user `87e05e88b9c44eeb9f7c4f2169c1f50b`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:41.882535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0c9edb16-2d62-4d08-ab2b-874c3d4518fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:41.921057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 815/1634] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:27:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:41.922388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.925882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:41.941156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a039d-1071-4836-a68b-608f7b0837c1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:41.952724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:41.954988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.955495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:41.955999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] PATH_INFO: `/v3/projects/7b58c19fc5d440b0a90ed88e91c0acd3/users/0b386c19ab754ffc8bfa314304058eed/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:41.959995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b58c19fc5d440b0a90ed88e91c0acd3, user_id=0b386c19ab754ffc8bfa314304058eed, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:41.965776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:41.982669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 816/1635] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:41 2022] PUT /identity/v3/projects/7b58c19fc5d440b0a90ed88e91c0acd3/users/0b386c19ab754ffc8bfa314304058eed/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:41.983787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:41.998195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 820/1636] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:41.999577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:41.999875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.000160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.039360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] MFA Rules not processed for user `0b386c19ab754ffc8bfa314304058eed`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:42.040564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:42.098283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e15b5-d53f-4232-a882-95fb6be22b4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:42.106824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:42.108726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.109448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.109781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.129903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:42.152937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 821/1637] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:42.153932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:42.173171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11f763-0a12-4d82-a816-7494bb9eeea8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:42.181326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:42.182021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.182303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.182654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.251143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:42.253628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:42.276692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 817/1638] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:42.301633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:42.303375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:42.330702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 822/1639] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:42.369380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:42.374510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:42.383891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351466f-2b53-422d-983c-368e02446ad8 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:42.388275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974ee13-e4a2-4b0e-9e95-e59bd9528036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:42.389321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974ee13-e4a2-4b0e-9e95-e59bd9528036 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.390211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974ee13-e4a2-4b0e-9e95-e59bd9528036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.390750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974ee13-e4a2-4b0e-9e95-e59bd9528036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.408950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974ee13-e4a2-4b0e-9e95-e59bd9528036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:42.411086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974ee13-e4a2-4b0e-9e95-e59bd9528036 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:42.440584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 818/1640] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:42.554555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:42.589286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798c14-bf7d-49df-913a-3959c66907e1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.589760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798c14-bf7d-49df-913a-3959c66907e1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.590134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88798c14-bf7d-49df-913a-3959c66907e1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.592384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 819/1641] 10.222.0.79 () {54 vars in 912 bytes} [Wed Jul 20 19:27:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:27:42.606311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.606701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.607331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.616873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7535e-19d8-47d4-9435-875d5993acba service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:42.627819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:42.629420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.629846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.630287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] PATH_INFO: `/v3/users/47cbc04374bb4c35af479290e62c8109` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.632446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47cbc04374bb4c35af479290e62c8109)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:42.638520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:42.651581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:42.653166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:42.673881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 820/1642] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:27:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:42.674732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:42.700368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18f3ae-942a-4eec-8c98-bbe0d735dfe9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:42.711491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d4212-8beb-46ce-90ea-75d8ce130c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:42.712017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d4212-8beb-46ce-90ea-75d8ce130c6c service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.712277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d4212-8beb-46ce-90ea-75d8ce130c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.712555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d4212-8beb-46ce-90ea-75d8ce130c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.732611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47cbc04374bb4c35af479290e62c8109'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:42.752036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47cbc04374bb4c35af479290e62c8109'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:42.773146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47cbc04374bb4c35af479290e62c8109'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:42.774190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47cbc04374bb4c35af479290e62c8109'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:42.788090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47cbc04374bb4c35af479290e62c8109'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:42.802090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 823/1643] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:42 2022] DELETE /identity/v3/users/47cbc04374bb4c35af479290e62c8109 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:42.808245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:42.808245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:42.856554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d4212-8beb-46ce-90ea-75d8ce130c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:42.858065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d4212-8beb-46ce-90ea-75d8ce130c6c service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:42.880525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 821/1644] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:42.882073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.882363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.882718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.912184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45da16-b379-4c8f-b235-0c702c99779f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:42.913104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:42.915046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:42.921058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:42.921943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:42.922377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:42.922710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:42.927112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:42.989106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 822/1645] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:43.080982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:43.083290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:43.093002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:43.106508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 824/1646] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:43.115072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:43.116280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:43.165700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:43.250820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137da99-6ad5-4d3a-856f-6d1324deced0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:43.251699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:43.253373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:43.259662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce425f20-e54b-47c7-9cf7-403155c58038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:43.260484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425f20-e54b-47c7-9cf7-403155c58038 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:43.260839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425f20-e54b-47c7-9cf7-403155c58038 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:43.261255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425f20-e54b-47c7-9cf7-403155c58038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:43.278944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104fec6-14de-41fc-8c01-60eec2dcba3d service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:43.284114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce425f20-e54b-47c7-9cf7-403155c58038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:43.285704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce425f20-e54b-47c7-9cf7-403155c58038 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:43.291963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:43.295933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:43.296564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:43.297263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] PATH_INFO: `/v3/users/b853c55a3b6f4f2cb7dc6dc68cb64826` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:43.299689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b853c55a3b6f4f2cb7dc6dc68cb64826)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:43.304037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:43.310659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 823/1647] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:43.421041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b853c55a3b6f4f2cb7dc6dc68cb64826'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:43.441169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b853c55a3b6f4f2cb7dc6dc68cb64826'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:43.470132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b853c55a3b6f4f2cb7dc6dc68cb64826'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:43.471152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b853c55a3b6f4f2cb7dc6dc68cb64826'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:43.486974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b853c55a3b6f4f2cb7dc6dc68cb64826'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:43.506141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 825/1648] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:43 2022] DELETE /identity/v3/users/b853c55a3b6f4f2cb7dc6dc68cb64826 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:43.512153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ae199-a332-430d-8677-cea0e954f7e0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:43.513368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ae199-a332-430d-8677-cea0e954f7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:43.514548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ae199-a332-430d-8677-cea0e954f7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:43.540377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5e8ae199-a332-430d-8677-cea0e954f7e0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:43.542571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5e8ae199-a332-430d-8677-cea0e954f7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:43.652218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 824/1649] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:43.686572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:43.694592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:43.829690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8131fd-7339-4f60-86fe-1ce969c05976 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:43.834222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:43.835017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:43.837257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:43.837690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:43.850278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:43.851997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:43.870387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 826/1650] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:44.671001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:44.671537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:44.672040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:44.735622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] MFA Rules not processed for user `9380df8196df4175b8d5e87f61a70e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:44.737419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:44.817314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:44.819256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:44.847989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62418a4d-117d-4612-a4a8-ed4193258937 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:44.859564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:44.860764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:44.861153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:44.861502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] PATH_INFO: `/v3/projects/01498bd6bd0848519576ff941722e05f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:44.863464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01498bd6bd0848519576ff941722e05f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:44.864541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: WARNING py.warnings [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:27:44.864541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: warnings.warn(msg) Jul 20 19:27:44.864541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: Jul 20 19:27:44.885185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:44.949683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 825/1651] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:44.950996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:44.953976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:44.978348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847fceb-8a45-4ef7-9fe9-2fe2979ffcaa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:45.017588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:45.017588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.017588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.017588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.017898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:45.017898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:45.063407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 826/1652] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:44 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:45.064402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:45.070961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:45.123580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01498bd6bd0848519576ff941722e05f', 'request_id': 'req-3825da39-3443-4f80-83cf-f74a14924096'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.125824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:45.140306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01498bd6bd0848519576ff941722e05f', 'request_id': 'req-3825da39-3443-4f80-83cf-f74a14924096'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.152584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01498bd6bd0848519576ff941722e05f', 'request_id': 'req-3825da39-3443-4f80-83cf-f74a14924096'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.183468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:45.197123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] The token cache is being invalidate because project 01498bd6bd0848519576ff941722e05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:45.198444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.199714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ff74d-4533-498a-839a-cd249a17f337 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:45.201366 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.222953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 827/1653] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:44 2022] DELETE /identity/v3/projects/01498bd6bd0848519576ff941722e05f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:45.224920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:45.229713 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:45.318666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:45.319957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.320633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.321093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] PATH_INFO: `/v3/projects/509e9c4cf31b4e9ea9d42a7bc4b99ac3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.323120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509e9c4cf31b4e9ea9d42a7bc4b99ac3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:45.327861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:45.392081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825da39-3443-4f80-83cf-f74a14924096 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:45.400832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:45.401867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.402304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.403213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.404703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:45.408622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:45.461749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 828/1654] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:45 2022] POST /identity/v3/users => generated 308 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:45.462848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:45.464321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509e9c4cf31b4e9ea9d42a7bc4b99ac3', 'request_id': 'req-70fee450-b5c3-4730-9b19-92c6866126c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.466782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:45.485529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509e9c4cf31b4e9ea9d42a7bc4b99ac3', 'request_id': 'req-70fee450-b5c3-4730-9b19-92c6866126c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.508110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509e9c4cf31b4e9ea9d42a7bc4b99ac3', 'request_id': 'req-70fee450-b5c3-4730-9b19-92c6866126c9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.516124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3898db7-4daf-48b1-86eb-53e14b54b0c2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:45.527410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] The token cache is being invalidate because project 509e9c4cf31b4e9ea9d42a7bc4b99ac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:45.527718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.529344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.533762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 827/1655] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:45 2022] DELETE /identity/v3/projects/509e9c4cf31b4e9ea9d42a7bc4b99ac3 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:45.535773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:45.538169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:45.672914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fee450-b5c3-4730-9b19-92c6866126c9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:45.684808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d307f1-e148-4886-98f1-bc53f91fff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:45.685531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d307f1-e148-4886-98f1-bc53f91fff59 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.685819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d307f1-e148-4886-98f1-bc53f91fff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.686100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d307f1-e148-4886-98f1-bc53f91fff59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.688775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:45.689589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.689872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.690148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] PATH_INFO: `/v3/users/87e05e88b9c44eeb9f7c4f2169c1f50b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.690366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d307f1-e148-4886-98f1-bc53f91fff59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:45.695588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e05e88b9c44eeb9f7c4f2169c1f50b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:45.700448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d307f1-e148-4886-98f1-bc53f91fff59 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:45.701730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:45.733403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 828/1656] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:45.733892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.734189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.734490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.752979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:45.793814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:45.799812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:45.838684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 829/1657] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:45.838955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e05e88b9c44eeb9f7c4f2169c1f50b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.839407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:45.854523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e05e88b9c44eeb9f7c4f2169c1f50b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.860170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25895f4a-73c8-4b9c-9297-b65a58e97a19 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:45.867428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:45.869299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.869712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.870102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] PATH_INFO: `/v3/projects/fab4740fb63e4031969c96f37bd5f689/users/9695e11729244489a153338df4372f8c/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.879636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e05e88b9c44eeb9f7c4f2169c1f50b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.880411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e05e88b9c44eeb9f7c4f2169c1f50b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.894743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e9e40cc6-2fdf-4fc1-a6cd-dc870d40676f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e05e88b9c44eeb9f7c4f2169c1f50b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:45.903821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab4740fb63e4031969c96f37bd5f689, user_id=9695e11729244489a153338df4372f8c, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:45.907633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:45.918382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 829/1658] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:45 2022] DELETE /identity/v3/users/87e05e88b9c44eeb9f7c4f2169c1f50b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:45.918581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:45.918764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:45.919053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:45.944210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:45.947185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 830/1659] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:45 2022] PUT /identity/v3/projects/fab4740fb63e4031969c96f37bd5f689/users/9695e11729244489a153338df4372f8c/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:45.956136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:45.959306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:45.966369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:45.967606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:46.116584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73340b-8125-4809-9cf3-e939eedf0485 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:46.116777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 830/1660] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:46.119158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:46.128573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:46.129587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:46.130397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:46.131895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:46.133445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:46.137751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:46.145626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c84be4-5fcb-4f84-98eb-41532556e5d6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:46.155191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 831/1661] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:46.157154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:46.167165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:46.169049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:46.169358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:46.169646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:46.171956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:46.173464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d4c96-9807-4112-911c-42c7fd705ebd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:46.173599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:46.178197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:46.179211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:46.182303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:46.182977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:46.183423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] PATH_INFO: `/v3/projects/fab4740fb63e4031969c96f37bd5f689/users/9695e11729244489a153338df4372f8c/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:46.185800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab4740fb63e4031969c96f37bd5f689, user_id=9695e11729244489a153338df4372f8c, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:46.191262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:46.213733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 831/1662] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:46.213967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 832/1663] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:46 2022] PUT /identity/v3/projects/fab4740fb63e4031969c96f37bd5f689/users/9695e11729244489a153338df4372f8c/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:46.214044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:46.229035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413afe9-e931-4586-ad80-a31c960aba49 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:46.344635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:46.346066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:46.346616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:46.348614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:46.350881 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:46.354876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:46.479963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 832/1664] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:46 2022] POST /identity/v3/projects => generated 371 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:46.486684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:46.539168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22feab8f-bb80-400a-b67d-0ab63c8b00f2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:46.756249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:46.757219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:46.757353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:46.758188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:46.758887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:46.760835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:46.823823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 833/1665] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:46 2022] POST /identity/v3/users => generated 286 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:46.824504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:46.863755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e31be8-f500-4cb7-930e-c2884ddf9289 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:46.885362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:46.888395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:46.914337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:46.915772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:46.917432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:46.919068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:46.922443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:46.931196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:46.985443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f147eec-51b5-4a8a-820b-017fd7db611f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:47.000989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 833/1666] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:47.006732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:47.011741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:47.019340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.020480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.021897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] PATH_INFO: `/v3/projects/bff9680748f64a15a9eba7110b8ea565` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.036808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff9680748f64a15a9eba7110b8ea565)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:47.045871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:47.052208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db130a02-a6a0-40d4-9a9e-bf6877cdf004 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:47.070889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:47.072484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.074383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.075039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.076532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:47.095159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:47.189161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 834/1667] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:27:47.190839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:47.215402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:47.272829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6a442-389f-4e67-8fbf-e644c0ac7827 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:47.331976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:47.356949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff9680748f64a15a9eba7110b8ea565', 'request_id': 'req-80653128-c5f9-43c3-a5a4-b049b3cb3631'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.375846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff9680748f64a15a9eba7110b8ea565', 'request_id': 'req-80653128-c5f9-43c3-a5a4-b049b3cb3631'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.396365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff9680748f64a15a9eba7110b8ea565', 'request_id': 'req-80653128-c5f9-43c3-a5a4-b049b3cb3631'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.441467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] The token cache is being invalidate because project bff9680748f64a15a9eba7110b8ea565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:47.441699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.444028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.447463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 834/1668] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:46 2022] DELETE /identity/v3/projects/bff9680748f64a15a9eba7110b8ea565 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:47.448481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:47.452222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:47.483782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:47.502429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:47.503351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.503635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.503914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] PATH_INFO: `/v3/users/c225586210fd4563bc61f2587961271b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.505216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c225586210fd4563bc61f2587961271b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:47.516999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:47.630750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80653128-c5f9-43c3-a5a4-b049b3cb3631 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:47.635099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c225586210fd4563bc61f2587961271b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.641476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4d5a5-cdc1-48e3-ae8c-d3f4cdc2a559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:47.642315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d5a5-cdc1-48e3-ae8c-d3f4cdc2a559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.642597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d5a5-cdc1-48e3-ae8c-d3f4cdc2a559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.642861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d5a5-cdc1-48e3-ae8c-d3f4cdc2a559 admin admin] PATH_INFO: `/v3/projects/79c54327a19e4dd8b81c4dd5c0c4096c/users/11ba25f4993645349c30131a4a51571c/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.655217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c225586210fd4563bc61f2587961271b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.677014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4d5a5-cdc1-48e3-ae8c-d3f4cdc2a559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c54327a19e4dd8b81c4dd5c0c4096c, user_id=11ba25f4993645349c30131a4a51571c, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:47.680620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4d5a5-cdc1-48e3-ae8c-d3f4cdc2a559 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:47.693189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c225586210fd4563bc61f2587961271b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.696416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c225586210fd4563bc61f2587961271b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.713184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 835/1669] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:47 2022] PUT /identity/v3/projects/79c54327a19e4dd8b81c4dd5c0c4096c/users/11ba25f4993645349c30131a4a51571c/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:47.715122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b93fb-6d9c-443d-9139-f120d1ca8cb6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.715644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b93fb-6d9c-443d-9139-f120d1ca8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.716122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b93fb-6d9c-443d-9139-f120d1ca8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.726727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-216ea806-4612-4633-bf23-4bcf5e37ad9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c225586210fd4563bc61f2587961271b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:47.745335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 835/1670] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:47 2022] DELETE /identity/v3/users/c225586210fd4563bc61f2587961271b => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:47.755439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.755733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.756018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.762105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bb8b93fb-6d9c-443d-9139-f120d1ca8cb6 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:47.765723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bb8b93fb-6d9c-443d-9139-f120d1ca8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:47.849364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] MFA Rules not processed for user `11ba25f4993645349c30131a4a51571c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:47.850873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:47.920947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 836/1671] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:47.924248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e73eb-2222-4599-a4a5-de90b241dee0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:47.924545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e73eb-2222-4599-a4a5-de90b241dee0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:47.924823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e73eb-2222-4599-a4a5-de90b241dee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:47.978591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c29e73eb-2222-4599-a4a5-de90b241dee0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:47.979624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c29e73eb-2222-4599-a4a5-de90b241dee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:48.004458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 837/1672] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:48.009298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.009692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.009961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.029704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 836/1673] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:48.031239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.042655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:48.089896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:48.092277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c039fe-576a-4bd6-90a5-50c9f108f1f3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.093744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:48.112441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:48.113692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.116183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.117105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.119354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:48.123306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:48.126422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 838/1674] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:48.127367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.130545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:48.185525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 837/1675] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:48 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:48.187715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.218017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:48.268816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe8f07-0b36-4710-bfb7-d5b004913d93 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.308941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:48.350686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db602d-aee8-49e2-9e0d-920155bdbac4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.365410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:48.371194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.371524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.371770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.374217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:48.374616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:48.376369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ead3f7-9bd1-47a9-bc8b-3650ba9b7691 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:48.397586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:48.398324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.398700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.399048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.414976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 838/1676] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:48.417137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.417482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.417772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.434280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:48.436014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:48.458191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 839/1677] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:48.461734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.475635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] MFA Rules not processed for user `9695e11729244489a153338df4372f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:48.477553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:48.503901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee1772-ce25-4511-836e-dca9b969cab0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.524501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:48.525810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.526464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.527006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.537377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:48.541686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:48.651054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 839/1678] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:48.651054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.651335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 840/1679] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:48 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:48.659267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.674053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef089f-002f-4be2-ba34-4bb8b3c2c5b6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.685758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:48.700800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.700800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.700800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.701717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:48.704394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:48.707850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3544-85ce-418d-854e-7b097aae549d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.738238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:48.739643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:48.740129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:48.740628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:48.742635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:48.745834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:48.765909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 840/1680] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:48 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:48.768136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.786670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 841/1681] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:48.793524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:48.839448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3e762-2d1a-46ee-a9d3-bebc266f1d2a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:48.975868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b027-77cf-4cca-a418-4df699a817d8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.001028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.002447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.003147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.003790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.005277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.009407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.096389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 841/1682] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:48 2022] POST /identity/v3/users => generated 290 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:49.098654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.103053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.104857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.106233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.113690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] PATH_INFO: `/v3/projects/e7b1549139e54022a4444bf2b22f27c4/users/29c27b8cff0549af8e53504d6cbdabe5/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.141841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910edd-1b1a-463d-8dfb-9f7c7a71c327 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.192984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b1549139e54022a4444bf2b22f27c4, user_id=29c27b8cff0549af8e53504d6cbdabe5, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.198024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.199449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.200061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.202377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.204089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.207896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.212233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.243806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 842/1683] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:48 2022] PUT /identity/v3/projects/e7b1549139e54022a4444bf2b22f27c4/users/29c27b8cff0549af8e53504d6cbdabe5/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:49.248159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.259086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 842/1684] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:49.266661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.286852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf554f-6e85-41d8-bc00-1293cbb34787 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.359748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:49.384259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:49.421897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91304ff-850f-4507-8f06-fee0e5e2de14 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.451130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.457409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.457964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.459069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.464868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.467800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.468516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.468800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.469102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.469852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.472056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.475646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.492986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 843/1685] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:49.499207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.504694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:49.516322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 843/1686] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:27:49.517322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.530207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaec9be-ba84-4bf7-9cac-830f2f8b3103 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.545881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0005a-8c4d-499a-9727-3297c3c53964 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.546937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.547830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.548114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.548390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] PATH_INFO: `/v3/projects/a2cb0c127060486b89ae4e2eb60bdbc6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.550258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cb0c127060486b89ae4e2eb60bdbc6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.552189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.562657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.563607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.564551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.564793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] PATH_INFO: `/v3/projects/722e2fdf07584c33a87015f2ee55204d/users/a37edc029d37487e97601c9cccf51f76/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.614747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722e2fdf07584c33a87015f2ee55204d, user_id=a37edc029d37487e97601c9cccf51f76, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.618206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:49.660163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 844/1687] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:49 2022] PUT /identity/v3/projects/722e2fdf07584c33a87015f2ee55204d/users/a37edc029d37487e97601c9cccf51f76/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:49.660163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.675724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cb0c127060486b89ae4e2eb60bdbc6', 'request_id': 'req-0c2223db-7037-40dc-a3c6-424f20706130'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:49.692709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cb0c127060486b89ae4e2eb60bdbc6', 'request_id': 'req-0c2223db-7037-40dc-a3c6-424f20706130'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:49.705626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cb0c127060486b89ae4e2eb60bdbc6', 'request_id': 'req-0c2223db-7037-40dc-a3c6-424f20706130'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:49.722731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] The token cache is being invalidate because project a2cb0c127060486b89ae4e2eb60bdbc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:49.723073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:49.724171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c2223db-7037-40dc-a3c6-424f20706130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:49.734158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 844/1688] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:49 2022] DELETE /identity/v3/projects/a2cb0c127060486b89ae4e2eb60bdbc6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:49.734158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.734158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.734158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.800584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] MFA Rules not processed for user `a37edc029d37487e97601c9cccf51f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:49.801844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:49.830626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de7133-713f-4d61-8fb9-6b9977063265 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:49.974751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:49.976314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 845/1689] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:49.977591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:49.979009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:49.980476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:49.981349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] PATH_INFO: `/v3/projects/e7b1549139e54022a4444bf2b22f27c4/users/29c27b8cff0549af8e53504d6cbdabe5/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:49.981605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:49.987899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b1549139e54022a4444bf2b22f27c4, user_id=29c27b8cff0549af8e53504d6cbdabe5, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:49.991534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:50.025627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa3df6-c592-4348-ad5e-48ddc049d4b4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:50.041376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 845/1690] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:49 2022] PUT /identity/v3/projects/e7b1549139e54022a4444bf2b22f27c4/users/29c27b8cff0549af8e53504d6cbdabe5/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:50.056296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:50.065533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:50.087677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:50.178933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:50.179880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.180236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.180490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] PATH_INFO: `/v3/users/0b386c19ab754ffc8bfa314304058eed` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:50.187945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b386c19ab754ffc8bfa314304058eed)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:50.199031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:50.219389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f4b48-ea4e-49f6-bfb4-727047559e00 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:50.234982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b4441-a87c-4f83-8cf0-d357c787c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:50.237632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b4441-a87c-4f83-8cf0-d357c787c9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.237927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b4441-a87c-4f83-8cf0-d357c787c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.238214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b4441-a87c-4f83-8cf0-d357c787c9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:50.240109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b4441-a87c-4f83-8cf0-d357c787c9ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:50.243691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b4441-a87c-4f83-8cf0-d357c787c9ed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:50.266368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:50.346653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b386c19ab754ffc8bfa314304058eed'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:50.364503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 846/1691] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:50 2022] POST /identity/v3/projects => generated 381 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:50.369796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b386c19ab754ffc8bfa314304058eed'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:50.375724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.376966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.377362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:50.430324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b386c19ab754ffc8bfa314304058eed'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:50.430750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] MFA Rules not processed for user `29c27b8cff0549af8e53504d6cbdabe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:50.432299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:50.433847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b386c19ab754ffc8bfa314304058eed'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:50.448569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b386c19ab754ffc8bfa314304058eed'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:50.466110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 846/1692] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:49 2022] DELETE /identity/v3/users/0b386c19ab754ffc8bfa314304058eed => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:50.467036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:50.478867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:50.625091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 847/1693] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:50.639183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:50.640597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:50.723353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd90b-bc5d-4e16-8af0-bda9efc6d381 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:50.741003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:50.742308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.742618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.742937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:50.743648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:50.746542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:50.785255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118915c-386a-4a7a-aed8-d060308f99c7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:50.807671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 847/1694] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:50 2022] POST /identity/v3/users => generated 297 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:50.812834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:50.841785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4b353-65d9-4a24-8793-4f5158ca3ddc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:50.848614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:50.849434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.849725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.850717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:50.851525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:50.853779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:50.869047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 848/1695] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:50.873410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:50.886877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de270b8f-1f63-4e87-a73d-06042f5fdfa9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:50.893775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:50.894805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.895168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.895522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] PATH_INFO: `/v3/projects/8f06da8718ed4c249bde3e46db4d8461/users/648701d1d7c742649f17776ea5d23f20/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:50.925476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f06da8718ed4c249bde3e46db4d8461, user_id=648701d1d7c742649f17776ea5d23f20, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:50.928642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:50.961768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 849/1696] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:50 2022] PUT /identity/v3/projects/8f06da8718ed4c249bde3e46db4d8461/users/648701d1d7c742649f17776ea5d23f20/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:50.965496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:50.971771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:50.979136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c24aae-2842-4c54-b63c-10601a72de00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:50.979808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c24aae-2842-4c54-b63c-10601a72de00 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:50.980103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c24aae-2842-4c54-b63c-10601a72de00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:50.980387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c24aae-2842-4c54-b63c-10601a72de00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:51.052209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfdb11-6029-4593-a4ef-fbb0437899e5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:51.058791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:51.059927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:51.060426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:51.061189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:51.062528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:51.064098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:51.066064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:51.083925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 850/1697] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:51.090086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:51.100857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c24aae-2842-4c54-b63c-10601a72de00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:51.103523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c24aae-2842-4c54-b63c-10601a72de00 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:51.106393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0d9e-41a7-4f8f-9924-eee72b4a35c3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:51.114438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:51.115412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:51.115710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:51.116547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] PATH_INFO: `/v3/projects/8f06da8718ed4c249bde3e46db4d8461/users/648701d1d7c742649f17776ea5d23f20/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:51.118696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 848/1698] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:51.119386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f06da8718ed4c249bde3e46db4d8461, user_id=648701d1d7c742649f17776ea5d23f20, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:51.122214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:51.157380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 851/1699] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:51 2022] PUT /identity/v3/projects/8f06da8718ed4c249bde3e46db4d8461/users/648701d1d7c742649f17776ea5d23f20/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:51.159309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211467-3184-472c-90da-720692652dd7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:51.160176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211467-3184-472c-90da-720692652dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:51.160632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211467-3184-472c-90da-720692652dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:51.211480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f4211467-3184-472c-90da-720692652dd7 None None] MFA Rules not processed for user `648701d1d7c742649f17776ea5d23f20`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:51.218333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4211467-3184-472c-90da-720692652dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:51.246454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:51.382284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a4a74-e168-45dd-b32b-5da5b12f691d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:51.392628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:51.394139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:51.394314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:51.394618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:51.401617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:51.453992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 849/1700] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:51.459647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211467-3184-472c-90da-720692652dd7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:51.462717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4211467-3184-472c-90da-720692652dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:51.489367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211467-3184-472c-90da-720692652dd7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:51.499106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:51.500037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:51.500141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:51.500344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:51.539926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:51.541566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:51.579177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 850/1701] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:27:51.580506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:51.581994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:51.597190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 852/1702] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:52.229610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:52.241419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f023ab-ec52-4088-9400-9939cd21bfa7 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:52.247607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '648701d1d7c742649f17776ea5d23f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f06da8718ed4c249bde3e46db4d8461', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:52.248555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:52.249216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:52.249956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:52.250903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:52.251402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:52.253262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:52.277152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 851/1703] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:52.281315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:52.298148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe167-465a-48a2-8dc7-e6517b87b7ae service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:52.304732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb79a7-0140-45c8-a3ee-92f307152efb tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '648701d1d7c742649f17776ea5d23f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f06da8718ed4c249bde3e46db4d8461', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:52.306085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb79a7-0140-45c8-a3ee-92f307152efb tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:52.312028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb79a7-0140-45c8-a3ee-92f307152efb tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:52.312564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb79a7-0140-45c8-a3ee-92f307152efb tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:52.313595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb79a7-0140-45c8-a3ee-92f307152efb tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:52.316267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb79a7-0140-45c8-a3ee-92f307152efb tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:52.353960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 853/1704] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:52 2022] POST /identity/v3/projects => generated 355 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:52.358836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:52.375318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55567c89-95bc-4578-876c-b8b695a95484 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:52.520405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '648701d1d7c742649f17776ea5d23f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f06da8718ed4c249bde3e46db4d8461', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:52.521605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:52.521902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:52.522184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:52.524558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:52.524971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:52.526606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:52.548330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 852/1705] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:52.617684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992911-e203-40ad-a40a-1756d44a76f0 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:52.617983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992911-e203-40ad-a40a-1756d44a76f0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:52.618267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992911-e203-40ad-a40a-1756d44a76f0 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:52.620231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:52.625056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 854/1706] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:27:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:27:52.634172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5931428-7bb7-4cea-85ff-2418cbe05785 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:52.654721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '648701d1d7c742649f17776ea5d23f20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f06da8718ed4c249bde3e46db4d8461', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:52.658331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:52.659362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:52.664682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] PATH_INFO: `/v3/projects/0e14ecb46dde44bcb04aed7dcf78256e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:52.667432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e14ecb46dde44bcb04aed7dcf78256e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:52.671021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:52.688354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992911-e203-40ad-a40a-1756d44a76f0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:52.690521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-15992911-e203-40ad-a40a-1756d44a76f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:52.727059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:52.751626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:52.755971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e14ecb46dde44bcb04aed7dcf78256e', 'request_id': 'req-4dc62828-dce9-4ef3-b65b-065e48db3cc3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:52.766124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e14ecb46dde44bcb04aed7dcf78256e', 'request_id': 'req-4dc62828-dce9-4ef3-b65b-065e48db3cc3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:52.781636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e14ecb46dde44bcb04aed7dcf78256e', 'request_id': 'req-4dc62828-dce9-4ef3-b65b-065e48db3cc3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:52.843778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] The token cache is being invalidate because project 0e14ecb46dde44bcb04aed7dcf78256e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:52.846010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:52.847318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4dc62828-dce9-4ef3-b65b-065e48db3cc3 tempest-VolumeQuotaClassesTest-1378184455 tempest-VolumeQuotaClassesTest-1378184455-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:52.861525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 853/1707] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:52 2022] DELETE /identity/v3/projects/0e14ecb46dde44bcb04aed7dcf78256e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:52.861525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8ee72-c01b-4337-9da2-c6986c14fab0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:52.861525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8ee72-c01b-4337-9da2-c6986c14fab0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:52.864524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8ee72-c01b-4337-9da2-c6986c14fab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:52.910583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b9d8ee72-c01b-4337-9da2-c6986c14fab0 None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:52.913368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9d8ee72-c01b-4337-9da2-c6986c14fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:52.934050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992911-e203-40ad-a40a-1756d44a76f0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:53.111407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:53.112961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.114447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.117872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] PATH_INFO: `/v3/projects/7b58c19fc5d440b0a90ed88e91c0acd3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.120383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b58c19fc5d440b0a90ed88e91c0acd3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:53.127939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:53.152743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 854/1708] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:53.241200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b58c19fc5d440b0a90ed88e91c0acd3', 'request_id': 'req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.247467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a86f2-c4ac-404f-a2b6-0e046ff5326f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.248009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a86f2-c4ac-404f-a2b6-0e046ff5326f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.248009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a86f2-c4ac-404f-a2b6-0e046ff5326f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.249764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 855/1709] 10.222.0.79 () {54 vars in 961 bytes} [Wed Jul 20 19:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:27:53.251639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b58c19fc5d440b0a90ed88e91c0acd3', 'request_id': 'req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.258725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.259125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.259302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.263092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b58c19fc5d440b0a90ed88e91c0acd3', 'request_id': 'req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.280417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] The token cache is being invalidate because project 7b58c19fc5d440b0a90ed88e91c0acd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:53.281113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.282288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10d3d3cf-77b9-495a-b37e-fe5e5bf5d546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.294446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 855/1710] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:52 2022] DELETE /identity/v3/projects/7b58c19fc5d440b0a90ed88e91c0acd3 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:53.296481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.297098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.297812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.316664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:53.317958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:53.326957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:53.330193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:53.483467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 856/1711] 10.222.0.79 () {58 vars in 1058 bytes} [Wed Jul 20 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:53.484343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:53.488571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:53.511323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:53.548656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 856/1712] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:53.548656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:53.569324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b357f9-ffc2-49ca-8ab8-4ee268d86e10 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:53.588261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:53.589466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.590046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.590357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.595702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:53.613115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:53.636785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:53.639811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfd23f-41ed-4c7d-abda-400b5abd7edd service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:53.647309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7d9e5-7b7b-4eab-9461-c3ec062f73ef None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:53.654330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:53.655517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.655902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.656301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] PATH_INFO: `/v3/users/a37edc029d37487e97601c9cccf51f76` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.659754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37edc029d37487e97601c9cccf51f76)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:53.662402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:27:53.662402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 20 19:27:53.662402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: Jul 20 19:27:53.666765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:53.677557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 857/1713] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:53.690277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.696080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.697611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.741218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:53.756232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:53.799977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 858/1714] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:53.806755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:53.847042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51719e-6d42-4f0a-b085-6a2459eaafce None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:53.862466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:53.866376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.869357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.869915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.871057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:27:53.871799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:53.875771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:53.877299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37edc029d37487e97601c9cccf51f76'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.911874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 859/1715] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:27:53.916276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:53.922063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37edc029d37487e97601c9cccf51f76'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:53.936291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ca90-f85b-448f-b867-c9c36b86f763 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:53.948359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:53.952312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:53.952684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:53.953158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:53.954001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:53.960068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:54.010067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37edc029d37487e97601c9cccf51f76'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:54.010886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37edc029d37487e97601c9cccf51f76'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:54.016542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 860/1716] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:53 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:54.019900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:54.032450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e1063f1-7c46-45aa-ad8c-90b56abbfa38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37edc029d37487e97601c9cccf51f76'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:54.041424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 857/1717] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:53 2022] DELETE /identity/v3/users/a37edc029d37487e97601c9cccf51f76 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:54.047250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.047799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.048092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.118351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:54.119689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:54.286224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404c176-162f-4726-a8b7-3af509c12a16 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:54.289455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 858/1718] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:54.297586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:54.298379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.298783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.300241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.300241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:54.302551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:54.333328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:54.335152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:54.348638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 861/1719] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:54 2022] POST /identity/v3/users => generated 285 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:54.353442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:54.359893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:54.410204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eebc58-13dc-41ea-a736-8cb6f460abc7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:54.425974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:54.427184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.427481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.427764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.431695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:54.433410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:54.474846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 862/1720] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:54.479295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:54.518821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c04db-ff38-44a6-abd6-9ff048a32284 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:54.531664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:54.532514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.532797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.542077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.546276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:54.551562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:54.560132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472dee8-11cb-424b-911b-cee9cf3f2c3f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:54.577835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7f016-b457-4c39-b4cf-caf1957a8db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:54.584829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f016-b457-4c39-b4cf-caf1957a8db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.585218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f016-b457-4c39-b4cf-caf1957a8db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.587743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f016-b457-4c39-b4cf-caf1957a8db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.599250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 863/1721] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:54.603644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:54.637688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7f016-b457-4c39-b4cf-caf1957a8db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:54.639562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7f016-b457-4c39-b4cf-caf1957a8db5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:54.649028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b137ea-37f1-444f-9f4d-38dad0483ade admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:54.676329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 859/1722] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:27:54.686665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:54.690074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.690468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.691064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] PATH_INFO: `/v3/projects/3d4e519facbc4e3cbc7cc660327a2e3e/users/7555330b214247c7a57e6d6627bf31e6/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.720740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4e519facbc4e3cbc7cc660327a2e3e, user_id=7555330b214247c7a57e6d6627bf31e6, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:54.724147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:54.767765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 864/1723] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:54 2022] PUT /identity/v3/projects/3d4e519facbc4e3cbc7cc660327a2e3e/users/7555330b214247c7a57e6d6627bf31e6/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:54.769833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:54.770140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:54.770402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:54.823813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:54.843674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] MFA Rules not processed for user `7555330b214247c7a57e6d6627bf31e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:54.844996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:55.031345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 860/1724] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:55.037510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:55.113265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:55.157044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe4455-b602-4763-b0e3-225227ee866d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:55.172013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:55.173230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.175599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.175903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.176694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:55.182411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:55.269293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 865/1725] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:27:55 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:55.271957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:55.300369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbeda6a-5e91-46a6-aa01-d339877e438e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:55.443047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:55.444137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.446611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.446984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.447749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:55.452220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:55.495526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 861/1726] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:27:55 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:27:55.501228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:55.517943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d5578-437a-4416-b588-fd582a991454 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:55.525427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:55.526131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.526422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.526706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.527649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:55.530246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:55.556704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 866/1727] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:55.558289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:55.572108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdaa0c-8f5c-4643-b965-9d7f29bb1847 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:55.577421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:55.578501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.578796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.579079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] PATH_INFO: `/v3/projects/b56116659e004045a0a2c2399bf362d8/users/6d0a33e59fc34a1990ba201e542cb720/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.612209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56116659e004045a0a2c2399bf362d8, user_id=6d0a33e59fc34a1990ba201e542cb720, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:55.617628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:55.651975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 862/1728] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:55 2022] PUT /identity/v3/projects/b56116659e004045a0a2c2399bf362d8/users/6d0a33e59fc34a1990ba201e542cb720/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:27:55.663809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:55.683115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7845e-3e79-4476-bb49-5cf576b65ce7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:55.808668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae15cc-299f-4cfb-9bbb-8e676668a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:55.809536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae15cc-299f-4cfb-9bbb-8e676668a29f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.809885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae15cc-299f-4cfb-9bbb-8e676668a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.810229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae15cc-299f-4cfb-9bbb-8e676668a29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.811044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae15cc-299f-4cfb-9bbb-8e676668a29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:55.812742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae15cc-299f-4cfb-9bbb-8e676668a29f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:55.840839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 867/1729] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:27:55.853422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:55.874431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c5681-bf20-448c-99a4-ef095d9c3371 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:55.880843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:55.882281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.882759 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.883385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] PATH_INFO: `/v3/projects/b56116659e004045a0a2c2399bf362d8/users/6d0a33e59fc34a1990ba201e542cb720/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.885950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56116659e004045a0a2c2399bf362d8, user_id=6d0a33e59fc34a1990ba201e542cb720, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:55.889476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:55.904194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 863/1730] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:27:55 2022] PUT /identity/v3/projects/b56116659e004045a0a2c2399bf362d8/users/6d0a33e59fc34a1990ba201e542cb720/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:55.916398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:55.916614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:55.917094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:55.973887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] MFA Rules not processed for user `6d0a33e59fc34a1990ba201e542cb720`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:55.978380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:56.167200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 868/1731] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:27:56.178665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:56.181488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:56.238532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:56.240024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:56.333539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:56.353858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509238f8-66a7-4856-a9a3-87abfcd95cb5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:56.382145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:56.385300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:56.386453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:56.386948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:56.390769 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a7d0-7a7b-4b72-be29-1a5479396ebf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:56.410925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:56.411849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:56.412117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:56.412491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] PATH_INFO: `/v3/users/9380df8196df4175b8d5e87f61a70e55` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:56.414342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9380df8196df4175b8d5e87f61a70e55)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:56.416414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:56.424780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:56.435168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:56.442612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:56.484622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 864/1732] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:56 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:56.518314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9380df8196df4175b8d5e87f61a70e55'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:56.537827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9380df8196df4175b8d5e87f61a70e55'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:56.571537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9380df8196df4175b8d5e87f61a70e55'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:56.573098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9380df8196df4175b8d5e87f61a70e55'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:56.594371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9380df8196df4175b8d5e87f61a70e55'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:56.614486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 869/1733] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:56 2022] DELETE /identity/v3/users/9380df8196df4175b8d5e87f61a70e55 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:56.899934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:56.901552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:56.927210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92adf6d-93dc-4ae1-b2f9-db22dcc35525 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:57.086450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:57.087748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:57.088188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:57.089617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] PATH_INFO: `/v3/projects/722e2fdf07584c33a87015f2ee55204d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:57.093134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722e2fdf07584c33a87015f2ee55204d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:57.098302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:57.227781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722e2fdf07584c33a87015f2ee55204d', 'request_id': 'req-f3e07cab-abcd-485d-966f-b0267d379bd9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.248907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722e2fdf07584c33a87015f2ee55204d', 'request_id': 'req-f3e07cab-abcd-485d-966f-b0267d379bd9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.282727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722e2fdf07584c33a87015f2ee55204d', 'request_id': 'req-f3e07cab-abcd-485d-966f-b0267d379bd9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.298330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] The token cache is being invalidate because project 722e2fdf07584c33a87015f2ee55204d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:57.298632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.301550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.309912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 865/1734] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:56 2022] DELETE /identity/v3/projects/722e2fdf07584c33a87015f2ee55204d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:57.310450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:57.312254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:57.329369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0aa23-d178-4ec8-beb0-1e9b2addc484 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:57.503194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:57.504101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:57.504382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:57.504658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] PATH_INFO: `/v3/users/648701d1d7c742649f17776ea5d23f20` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:57.506155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=648701d1d7c742649f17776ea5d23f20)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:57.513291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:57.626854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648701d1d7c742649f17776ea5d23f20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.641506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648701d1d7c742649f17776ea5d23f20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.653305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:57.677931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648701d1d7c742649f17776ea5d23f20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.679818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648701d1d7c742649f17776ea5d23f20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.698350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-223b3abc-b1c7-4ac3-b6fd-99f5fdee4eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648701d1d7c742649f17776ea5d23f20'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:57.719350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 870/1735] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:57 2022] DELETE /identity/v3/users/648701d1d7c742649f17776ea5d23f20 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:27:58.252409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:58.253957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:58.277551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2880b90-6877-43d8-a82b-23e65312f6d3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:58.277840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2880b90-6877-43d8-a82b-23e65312f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:58.278177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2880b90-6877-43d8-a82b-23e65312f6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:58.291185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e07cab-abcd-485d-966f-b0267d379bd9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:58.386198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e2880b90-6877-43d8-a82b-23e65312f6d3 None None] MFA Rules not processed for user `a271ea90c6884817991aaa9e8be36cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:58.390296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e2880b90-6877-43d8-a82b-23e65312f6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:58.415341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:58.416249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:58.416539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:58.416820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] PATH_INFO: `/v3/projects/414eac1ab7eb41f284cb191a2388a398` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:58.418791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] RBAC: Authorizing `identity:delete_project(project_id=414eac1ab7eb41f284cb191a2388a398)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:58.424723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:58.508352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:58.560960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414eac1ab7eb41f284cb191a2388a398', 'request_id': 'req-e9062274-57bf-4b54-b743-a6a26dc5004b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:58.571111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414eac1ab7eb41f284cb191a2388a398', 'request_id': 'req-e9062274-57bf-4b54-b743-a6a26dc5004b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:58.581732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414eac1ab7eb41f284cb191a2388a398', 'request_id': 'req-e9062274-57bf-4b54-b743-a6a26dc5004b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:58.598885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] The token cache is being invalidate because project 414eac1ab7eb41f284cb191a2388a398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:27:58.599196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:58.600960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:58.603312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 866/1736] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:58 2022] DELETE /identity/v3/projects/414eac1ab7eb41f284cb191a2388a398 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:58.608234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:58.613193 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:58.745534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 871/1737] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:58.757571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:58.757880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:58.758149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:58.815038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9062274-57bf-4b54-b743-a6a26dc5004b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:58.822420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:58.823503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:58.823926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:58.824307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] PATH_INFO: `/v3/users/9695e11729244489a153338df4372f8c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:58.826063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9695e11729244489a153338df4372f8c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:58.829037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] MFA Rules not processed for user `a271ea90c6884817991aaa9e8be36cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:58.829370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:58.834621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:58.985774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9695e11729244489a153338df4372f8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:59.012892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9695e11729244489a153338df4372f8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:59.036271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 872/1738] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:59.048323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:59.051548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:59.077615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9695e11729244489a153338df4372f8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:59.079435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9695e11729244489a153338df4372f8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:59.109416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6b20e0cb-5b16-46b3-9471-229cb1116387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9695e11729244489a153338df4372f8c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:59.120771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 867/1739] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:58 2022] DELETE /identity/v3/users/9695e11729244489a153338df4372f8c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:27:59.253256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa76db-385b-4824-9540-710db5516553 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:59.253877 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa76db-385b-4824-9540-710db5516553 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:59.255144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa76db-385b-4824-9540-710db5516553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:59.282236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078c070-41b2-461d-85a2-d841fa203a42 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:59.326995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:59.370285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-79aa76db-385b-4824-9540-710db5516553 None None] MFA Rules not processed for user `3f7758d298f642138b71aa422d0344da`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:27:59.371306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-79aa76db-385b-4824-9540-710db5516553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:59.495419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:59.564932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:59.565708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:59.566689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:59.567876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:59.570645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:59.603622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 868/1740] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:27:59.604575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa76db-385b-4824-9540-710db5516553 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:59.606060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-79aa76db-385b-4824-9540-710db5516553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:59.758156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:59.761831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:59.763435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa76db-385b-4824-9540-710db5516553 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:59.775680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:59.776767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:59.777109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:59.777443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] PATH_INFO: `/v3/users/7555330b214247c7a57e6d6627bf31e6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:59.778832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7555330b214247c7a57e6d6627bf31e6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:59.784129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:59.805413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 873/1741] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:27:59.819123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:27:59.836130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:27:59.876395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b2420-f8b8-4755-a62c-d51b676830e0 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:27:59.891514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:27:59.892419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:27:59.892705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:27:59.893010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] PATH_INFO: `/v3/projects/fab4740fb63e4031969c96f37bd5f689` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:27:59.896103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab4740fb63e4031969c96f37bd5f689)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:27:59.899500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:27:59.943867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7555330b214247c7a57e6d6627bf31e6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:27:59.982277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7555330b214247c7a57e6d6627bf31e6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.045293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7555330b214247c7a57e6d6627bf31e6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.047897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7555330b214247c7a57e6d6627bf31e6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.055726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab4740fb63e4031969c96f37bd5f689', 'request_id': 'req-5414458d-318b-448a-9df7-35858e2331bb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.064931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab4740fb63e4031969c96f37bd5f689', 'request_id': 'req-5414458d-318b-448a-9df7-35858e2331bb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.080016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab4740fb63e4031969c96f37bd5f689', 'request_id': 'req-5414458d-318b-448a-9df7-35858e2331bb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.093394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7555330b214247c7a57e6d6627bf31e6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.102753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] The token cache is being invalidate because project fab4740fb63e4031969c96f37bd5f689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:00.102997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.112112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.117459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 874/1742] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:27:59 2022] DELETE /identity/v3/projects/fab4740fb63e4031969c96f37bd5f689 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:00.118473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:00.119968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:00.126521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 869/1743] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:27:59 2022] DELETE /identity/v3/users/7555330b214247c7a57e6d6627bf31e6 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:00.131717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:00.137548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:00.220962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:00.291531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:00.339650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa929a8-5dc0-4a35-8922-90493f304033 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:00.348060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414458d-318b-448a-9df7-35858e2331bb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:00.357800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:00.360772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:00.364062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:00.364399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] PATH_INFO: `/v3/projects/8f06da8718ed4c249bde3e46db4d8461` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:00.370637 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f06da8718ed4c249bde3e46db4d8461)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:00.372703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:00.575179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f06da8718ed4c249bde3e46db4d8461', 'request_id': 'req-32dfadd7-8664-4bb8-9172-bbad424616b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.603200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f06da8718ed4c249bde3e46db4d8461', 'request_id': 'req-32dfadd7-8664-4bb8-9172-bbad424616b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.640517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f06da8718ed4c249bde3e46db4d8461', 'request_id': 'req-32dfadd7-8664-4bb8-9172-bbad424616b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.646614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:00.658212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:00.658754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:00.659061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:00.667013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:00.719804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] The token cache is being invalidate because project 8f06da8718ed4c249bde3e46db4d8461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:00.723682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.731930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-32dfadd7-8664-4bb8-9172-bbad424616b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:00.734031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 870/1744] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:00 2022] DELETE /identity/v3/projects/8f06da8718ed4c249bde3e46db4d8461 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:00.735996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc6e2-bd1a-4f23-aa9d-862cc1d6e0a8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:00.743130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc6e2-bd1a-4f23-aa9d-862cc1d6e0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:00.743590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc6e2-bd1a-4f23-aa9d-862cc1d6e0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:00.794401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fc2dc6e2-bd1a-4f23-aa9d-862cc1d6e0a8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:00.809059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fc2dc6e2-bd1a-4f23-aa9d-862cc1d6e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:00.881895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:00.884641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58df7e5-e683-42c1-919a-205b8688d712 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:00.931040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 875/1745] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 813 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:00.932702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348433ef-cdd2-48e4-ae87-113c9d8eb68a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:00.933203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348433ef-cdd2-48e4-ae87-113c9d8eb68a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:00.933833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348433ef-cdd2-48e4-ae87-113c9d8eb68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:00.996874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-348433ef-cdd2-48e4-ae87-113c9d8eb68a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:01.008146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-348433ef-cdd2-48e4-ae87-113c9d8eb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.213963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 876/1746] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:01.223685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700269c-4169-4e59-8da7-400800b977e3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.224184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700269c-4169-4e59-8da7-400800b977e3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.224573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700269c-4169-4e59-8da7-400800b977e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.247587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 871/1747] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:01.249114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43b741-ca4c-4565-a1a6-75ea051cf667 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.249416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43b741-ca4c-4565-a1a6-75ea051cf667 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.249700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43b741-ca4c-4565-a1a6-75ea051cf667 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.252811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 872/1748] 10.222.0.79 () {54 vars in 912 bytes} [Wed Jul 20 19:28:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:28:01.253585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43b741-ca4c-4565-a1a6-75ea051cf667 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:01.262052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:01.281847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8700269c-4169-4e59-8da7-400800b977e3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:01.283303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8700269c-4169-4e59-8da7-400800b977e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.299412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d43b741-ca4c-4565-a1a6-75ea051cf667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.300939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 877/1749] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:01.302875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045e405-c872-4ddd-8a2c-7436d392f7a8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.310448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045e405-c872-4ddd-8a2c-7436d392f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.311438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045e405-c872-4ddd-8a2c-7436d392f7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.398518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0045e405-c872-4ddd-8a2c-7436d392f7a8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:01.400016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0045e405-c872-4ddd-8a2c-7436d392f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.442709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 878/1750] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:01.444109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.446376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.446781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.491435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43b741-ca4c-4565-a1a6-75ea051cf667 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:01.500508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24405399-7ca0-4cab-8f49-2e61a5dbc129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:01.501479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24405399-7ca0-4cab-8f49-2e61a5dbc129 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.501770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24405399-7ca0-4cab-8f49-2e61a5dbc129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.502048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24405399-7ca0-4cab-8f49-2e61a5dbc129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.530380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24405399-7ca0-4cab-8f49-2e61a5dbc129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:01.532283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24405399-7ca0-4cab-8f49-2e61a5dbc129 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:01.563830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 873/1751] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:01.565874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.566173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.566461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.636490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:01.638362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:01.639723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.640599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.666916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 874/1752] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:01.669105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:01.671849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 879/1753] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:01.675875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:01.695249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2983b4e-ba3f-40de-b69a-5f13d6a3316f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:01.711749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:01.713763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.714183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.714578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.716815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:01.717336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:01.732635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010b338-54d4-444b-aaf3-eadc618aaf94 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:01.739759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:01.740494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.741065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.741349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.742346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:01.742558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:01.744469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dd2d7-3b4a-4e6d-b538-5cc71fca9e94 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:01.747790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:01.795483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:01.800906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 880/1754] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:01.804032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.806270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.806909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.818364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 875/1755] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:01.818364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:01.864243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae4873-4035-45f0-a5ac-ab4423b9fb89 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:01.890303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:01.891063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:01.891349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:01.891708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:01.892319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:01.894288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:01.909016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] MFA Rules not processed for user `c18c8d4034ce40dba9f9f48e96a19814`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:01.910782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:01.992527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 876/1756] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:01 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:01.998492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:02.109422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 881/1757] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:02.113875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:02.143904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c99f4-8369-405b-918b-e4cd16a1ea62 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:02.246981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b73fd-2a6b-465c-8a23-983d165debb8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:02.268991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:02.270221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:02.274126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:02.275222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:02.313143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:02.317368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:02.347743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 877/1758] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:02.353949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:02.433145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:02.434081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:02.435488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:02.435783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:02.437884 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:02.446012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:02.530161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 882/1759] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:02 2022] POST /identity/v3/projects => generated 385 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:02.530161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:02.535492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:02.587161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25b07e-3091-4e9e-9fe8-513467673a1d service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:02.659156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:02.730560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:02.731273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:02.731560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:02.731839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:02.732554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:02.737447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:02.781910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 878/1760] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:02 2022] POST /identity/v3/users => generated 291 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:02.782901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:02.802747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3043797-0ab2-4971-b27c-ff31d8cf72e2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:02.825686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:02.826390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:02.826678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:02.826956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:02.827658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:02.829701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:02.830629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a308ed-de2b-4c2d-9d63-3dc46063c16c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:02.853633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:02.854527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:02.854835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:02.855109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:02.859283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:02.872700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 879/1761] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:02 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:02.873903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:02.914711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ec35-7133-4052-8a51-099e94d02b3d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:02.931784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:02.936382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:02.938115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:02.938419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:02.939183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:02.947889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:02.983053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 880/1762] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:02.985497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:03.009907 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8149fd-4b7b-4de6-9e1d-4b03876f7d22 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:03.043506 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:03.044202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.044491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.044773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.050736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:03.053137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:03.056494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:03.058423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:03.093178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 881/1763] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:03.094144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:03.100206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 883/1764] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:03.101928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:03.124814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a054-e168-473c-9f88-09367e5c5d6a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:03.130943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023e70-6075-48da-8bce-2749b6627bda service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:03.138659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6513514-3997-48e5-8833-e08577bd2922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:03.139357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6513514-3997-48e5-8833-e08577bd2922 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.150821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6513514-3997-48e5-8833-e08577bd2922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.152101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6513514-3997-48e5-8833-e08577bd2922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.154719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6513514-3997-48e5-8833-e08577bd2922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:03.155143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:03.157678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.157985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.158986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6513514-3997-48e5-8833-e08577bd2922 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:03.159384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.160159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:03.166728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:03.188638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 882/1765] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:03.190148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38682d2-fcb0-45a7-929d-183627d3b5c1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.190499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38682d2-fcb0-45a7-929d-183627d3b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.190712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38682d2-fcb0-45a7-929d-183627d3b5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.205542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 884/1766] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:03.206494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:03.237040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2759fa5-d578-4c96-a1f2-03b85f3b835c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:03.255036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:03.255955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.256392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.256789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] PATH_INFO: `/v3/projects/b28889e9b7f44903b09e87c964765d4b/users/a0469e93b5504705b1c4ea0641f4499b/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.271871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e38682d2-fcb0-45a7-929d-183627d3b5c1 None None] MFA Rules not processed for user `347106fdd7c94624ab9d40b51da78623`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:03.274361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e38682d2-fcb0-45a7-929d-183627d3b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:03.293720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28889e9b7f44903b09e87c964765d4b, user_id=a0469e93b5504705b1c4ea0641f4499b, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:03.320588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:03.365318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 885/1767] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:03 2022] PUT /identity/v3/projects/b28889e9b7f44903b09e87c964765d4b/users/a0469e93b5504705b1c4ea0641f4499b/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:03.374302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:03.529838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 883/1768] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:03.542275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e60f2f-30b4-40e6-8ded-78f6009283b6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.542635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e60f2f-30b4-40e6-8ded-78f6009283b6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.542924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e60f2f-30b4-40e6-8ded-78f6009283b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.626956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b7e60f2f-30b4-40e6-8ded-78f6009283b6 None None] MFA Rules not processed for user `a0469e93b5504705b1c4ea0641f4499b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:03.638230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b7e60f2f-30b4-40e6-8ded-78f6009283b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:03.641876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9654514-11ef-4afc-bee5-20c33c71afdc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:03.657538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04c9bb-235b-49d4-815d-2641922ea141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:03.661019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04c9bb-235b-49d4-815d-2641922ea141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.663151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04c9bb-235b-49d4-815d-2641922ea141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.664308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04c9bb-235b-49d4-815d-2641922ea141 admin admin] PATH_INFO: `/v3/projects/b45df66c0ffd4dda8c342be251a91565/users/7bb70e74e54940ec9f29e9f656fe721e/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.706449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04c9bb-235b-49d4-815d-2641922ea141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45df66c0ffd4dda8c342be251a91565, user_id=7bb70e74e54940ec9f29e9f656fe721e, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:03.711265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04c9bb-235b-49d4-815d-2641922ea141 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:03.766638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 886/1769] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:03 2022] PUT /identity/v3/projects/b45df66c0ffd4dda8c342be251a91565/users/7bb70e74e54940ec9f29e9f656fe721e/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:03.778150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.778343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.778627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.844532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] MFA Rules not processed for user `347106fdd7c94624ab9d40b51da78623`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:03.868741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:03.888936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 884/1770] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:03.895686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:03.896169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:03.896611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:03.968294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:03.977343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] MFA Rules not processed for user `7bb70e74e54940ec9f29e9f656fe721e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:03.983732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.067497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:04.130402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 887/1771] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:04.131281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:04.134126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.168913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 885/1772] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:04.170778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:04.171332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.315737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48d8e9-6ced-495c-b746-bc6ddb43b473 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:04.316280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f0d57-6dad-495f-a729-904c6d7e293c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:04.323036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:04.324249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:04.324791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:04.325324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:04.328717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:04.329645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:04.330044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:04.330254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] PATH_INFO: `/v3/projects/3d4e519facbc4e3cbc7cc660327a2e3e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:04.344346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4e519facbc4e3cbc7cc660327a2e3e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:04.351688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:04.448002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:04.450139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:04.474282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 886/1773] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:04.476807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:04.480568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.487533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4e519facbc4e3cbc7cc660327a2e3e', 'request_id': 'req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:04.502707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4e519facbc4e3cbc7cc660327a2e3e', 'request_id': 'req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:04.518683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4e519facbc4e3cbc7cc660327a2e3e', 'request_id': 'req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:04.567283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] The token cache is being invalidate because project 3d4e519facbc4e3cbc7cc660327a2e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:04.567586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:04.574578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:04.578785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 888/1774] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:04 2022] DELETE /identity/v3/projects/3d4e519facbc4e3cbc7cc660327a2e3e => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:04.579687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:04.581786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.703137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46add9fa-eb0e-4e2c-80d1-16d4504dd5db service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:04.756804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:04.802339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:04.811059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ca81a-4e8a-461e-93a3-65d6ada6b5ba admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:04.830574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:04.831273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:04.833037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:04.833154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:04.839021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.886204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:04.889047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:04.889338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:04.889614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:04.896427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:04.937450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:04.942717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:04.976190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 887/1775] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:04.977706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:04.982882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:05.010248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:05.013739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:05.040332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 889/1776] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:05.041061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:05.059787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:05.271693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baad19e-9f8f-4884-86ab-3cbbfd9cf8d7 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:05.289755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:05.296285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:05.296285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:05.296285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:05.296285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:05.300695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:05.381347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 888/1777] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:04 2022] POST /identity/v3/projects => generated 385 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:05.383615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:05.479954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484fd3-d5a7-4a01-b070-2cd84c70951f service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:05.564029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338564e4-bc37-4848-b464-0c9f394deca4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:05.590577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:05.599875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:05.600222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:05.600507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:05.606534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:05.616766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:05.707644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 889/1778] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:05 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:05.712633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:05.738406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9164523-091a-41fd-9b2d-b3353c62e693 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:05.746192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:05.748236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:05.748691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:05.748801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] PATH_INFO: `/v3/users/6d0a33e59fc34a1990ba201e542cb720` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:05.750379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0a33e59fc34a1990ba201e542cb720)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:05.754175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:05.754770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:05.755421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:05.755734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:05.758738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:05.759892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:05.762844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:05.796253 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 890/1779] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:05.800959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:05.848087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:05.859512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24d24-81cb-4767-baf5-e23ba3fe2d0c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:05.859974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0a33e59fc34a1990ba201e542cb720'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:05.866533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:05.867626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:05.867969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:05.868259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:05.869056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:05.870918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:05.883881 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0a33e59fc34a1990ba201e542cb720'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:05.910067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 891/1780] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:05.917266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:05.958540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0a33e59fc34a1990ba201e542cb720'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:05.961986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0a33e59fc34a1990ba201e542cb720'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:05.970730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3080a87-3c04-4fde-8c1c-7447935c00dc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:05.981698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-762a61ae-fe48-4ec1-a345-9e599ad10281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0a33e59fc34a1990ba201e542cb720'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:05.985942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:05.995413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:05.996056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:05.996565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] PATH_INFO: `/v3/projects/637ef0b4b7dd4c72a3ba3d42df740fe4/users/73aeaaeb57734ea08ca1074efe11b643/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:06.001552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 890/1781] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:05 2022] DELETE /identity/v3/users/6d0a33e59fc34a1990ba201e542cb720 => generated 0 bytes in 958 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:06.049348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:06.061147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637ef0b4b7dd4c72a3ba3d42df740fe4, user_id=73aeaaeb57734ea08ca1074efe11b643, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:06.066128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:06.109832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 892/1782] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:05 2022] PUT /identity/v3/projects/637ef0b4b7dd4c72a3ba3d42df740fe4/users/73aeaaeb57734ea08ca1074efe11b643/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:06.123706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:06.124003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:06.124304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:06.217056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] MFA Rules not processed for user `73aeaaeb57734ea08ca1074efe11b643`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:06.221117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:06.449574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 891/1783] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:06.450607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:06.454170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:06.689252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5532cb2-37d6-44fb-a43e-e68df74a8909 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:06.715629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:06.721178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:06.721885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:06.724273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:06.725252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:06.743002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:06.838636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 893/1784] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:06 2022] POST /identity/v3/projects => generated 387 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:06.840349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:06.868447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dded1-c5e9-4c17-b4f1-abe9d8c9eaba None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:07.082224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:07.084264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.085929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.087466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:07.090944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:07.094886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:07.155090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 892/1785] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:06 2022] POST /identity/v3/users => generated 300 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:07.159412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:07.178174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16f67-988c-46b9-b357-d89370e82dc9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:07.196347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:07.197648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.198971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.199257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:07.204190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:07.212307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:07.244703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:07.246826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 894/1786] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:07.265007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f26b5c-a6b7-442e-9772-460d3d1d0765 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:07.281773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:07.283266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.284816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.285538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] PATH_INFO: `/v3/projects/bbc172c2cbb94b3e8fe205cbee41b1f8/users/25d284b36ddc448fb582f6a74568b5e3/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:07.327894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc172c2cbb94b3e8fe205cbee41b1f8, user_id=25d284b36ddc448fb582f6a74568b5e3, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:07.331104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:07.361743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 893/1787] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:07 2022] PUT /identity/v3/projects/bbc172c2cbb94b3e8fe205cbee41b1f8/users/25d284b36ddc448fb582f6a74568b5e3/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:07.363000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:07.380474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae107976-4609-4063-929c-aaa7bb7f72f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:07.480332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34d376-e264-4d5e-bd47-f197070bb58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:07.481299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d376-e264-4d5e-bd47-f197070bb58a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.481703 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d376-e264-4d5e-bd47-f197070bb58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.482096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d376-e264-4d5e-bd47-f197070bb58a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:07.483025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d376-e264-4d5e-bd47-f197070bb58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:07.485470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d376-e264-4d5e-bd47-f197070bb58a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:07.508262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:07.519860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 895/1788] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:07.544291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f6f5b-170b-4079-a109-b3b17bc479d5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:07.565979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:07.567427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.568415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.568595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] PATH_INFO: `/v3/projects/bbc172c2cbb94b3e8fe205cbee41b1f8/users/25d284b36ddc448fb582f6a74568b5e3/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:07.571131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc172c2cbb94b3e8fe205cbee41b1f8, user_id=25d284b36ddc448fb582f6a74568b5e3, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:07.574637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:07.592371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:07.598812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 894/1789] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:07 2022] PUT /identity/v3/projects/bbc172c2cbb94b3e8fe205cbee41b1f8/users/25d284b36ddc448fb582f6a74568b5e3/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:07.614964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0275c-6efc-4347-9d05-90389a74a29e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.615417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0275c-6efc-4347-9d05-90389a74a29e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.615774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0275c-6efc-4347-9d05-90389a74a29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:07.690974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a6d0275c-6efc-4347-9d05-90389a74a29e None None] MFA Rules not processed for user `25d284b36ddc448fb582f6a74568b5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:07.692192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a6d0275c-6efc-4347-9d05-90389a74a29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:07.830558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:07.878013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 896/1790] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:07.887865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:07.889687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:07.979428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dd449-b80e-4ab1-b03d-607246604661 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:07.979732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dd449-b80e-4ab1-b03d-607246604661 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:07.980019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dd449-b80e-4ab1-b03d-607246604661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:08.024613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d55dd449-b80e-4ab1-b03d-607246604661 None None] MFA Rules not processed for user `3f7758d298f642138b71aa422d0344da`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:08.025715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d55dd449-b80e-4ab1-b03d-607246604661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:08.093119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175fb39-2130-43bf-9cee-b4c1b231a86a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:08.100615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:08.105105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:08.105687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:08.107950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:08.132487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:08.134960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:08.176171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 895/1791] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:08.177934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:08.181587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 897/1792] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:08.182025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:08.387028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73360a22-4aa4-4329-993e-8b9c602cc126 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:08.397963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:08.399408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:08.399701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:08.400005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] PATH_INFO: `/v3/projects/b56116659e004045a0a2c2399bf362d8` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:08.427966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56116659e004045a0a2c2399bf362d8)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:08.439586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:08.579987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56116659e004045a0a2c2399bf362d8', 'request_id': 'req-f91a6462-2192-444c-9906-96a31a1f7eeb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:08.608454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56116659e004045a0a2c2399bf362d8', 'request_id': 'req-f91a6462-2192-444c-9906-96a31a1f7eeb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:08.625806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56116659e004045a0a2c2399bf362d8', 'request_id': 'req-f91a6462-2192-444c-9906-96a31a1f7eeb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:08.657349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] The token cache is being invalidate because project b56116659e004045a0a2c2399bf362d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:08.657685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:08.659016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f91a6462-2192-444c-9906-96a31a1f7eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:08.674031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 896/1793] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:08 2022] DELETE /identity/v3/projects/b56116659e004045a0a2c2399bf362d8 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:08.678745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5f4fe-2e0f-4a54-b0e3-7b3e1889572c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:08.679128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5f4fe-2e0f-4a54-b0e3-7b3e1889572c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:08.679399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5f4fe-2e0f-4a54-b0e3-7b3e1889572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:08.724505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8ac5f4fe-2e0f-4a54-b0e3-7b3e1889572c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:08.743979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8ac5f4fe-2e0f-4a54-b0e3-7b3e1889572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:08.774619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:08.774913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:08.775193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:08.838562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] MFA Rules not processed for user `347106fdd7c94624ab9d40b51da78623`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:08.840573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:08.947303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:08.979706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 898/1794] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:08.989146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:08.989309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:08.989581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.052279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 897/1795] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:09.052490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:09.052490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:09.100139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 899/1796] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:09.106381 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.127693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec34df-fd4f-4e44-b050-80148b64f5e1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.143868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.144580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.144997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.145170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.146039 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:09.146348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:09.148206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec8d1-592a-44b2-95c4-88db02c0a632 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:09.209711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 898/1797] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:09.214665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.226451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607372c-6571-496e-a3c7-55baba508cdd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.245858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.246777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.247175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.247560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.248383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:09.250458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:09.340943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579d33-0c5c-42a5-b82d-e21d1a4c6149 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.341118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579d33-0c5c-42a5-b82d-e21d1a4c6149 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.341434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579d33-0c5c-42a5-b82d-e21d1a4c6149 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.343232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 899/1798] 10.222.0.79 () {54 vars in 912 bytes} [Wed Jul 20 19:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:28:09.362803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.363132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.363469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.404694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 900/1799] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:09 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:09.406974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.454943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:09.458199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:09.643171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 900/1800] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:28:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:09.644394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.650173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:09.683889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e758ddb-bf13-4de5-aa6f-c5a04ee4705d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.691562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.692428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.692733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.693079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.694050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:09.698634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:09.712925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:09.751622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 901/1801] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:09 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:09.752589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.754965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:09.791709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ec23-a611-4918-8965-5ad811fdaef0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.801793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.807074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.808620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.809630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.815175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460a5ed-7a68-4715-b32f-379ac5fc7140 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.827116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:09.829795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:09.831024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.832360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.832360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.843721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 902/1802] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:09.844238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.845992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.847453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:09.864187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3321-199e-4558-a27b-115ea7bfb580 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.896340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.897179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.897560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.897830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.899965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:09.902232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:09.925617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 903/1803] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:09.930748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:09.962204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602b1eb-4c82-4fd0-a63f-9738af4645ea admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:09.973727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:09.974840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:09.975540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:09.977164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:09.978606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:09.980706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:10.002433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 904/1804] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:10.009526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:10.053240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0d30-2946-4f3e-80a2-7c235ec2ea9d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:10.079242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24fc38-ed44-4a8d-b8c1-38b7a86cad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:10.080655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fc38-ed44-4a8d-b8c1-38b7a86cad12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:10.081004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fc38-ed44-4a8d-b8c1-38b7a86cad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:10.081704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fc38-ed44-4a8d-b8c1-38b7a86cad12 admin admin] PATH_INFO: `/v3/projects/5b79c86cbfb14712a4b234130583f9f2/users/0a5918073e9c402a8b642c5897dd8e87/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:10.108401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fc38-ed44-4a8d-b8c1-38b7a86cad12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b79c86cbfb14712a4b234130583f9f2, user_id=0a5918073e9c402a8b642c5897dd8e87, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:10.115541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:10.118842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fc38-ed44-4a8d-b8c1-38b7a86cad12 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:10.120693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:10.159357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 905/1805] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:10 2022] PUT /identity/v3/projects/5b79c86cbfb14712a4b234130583f9f2/users/0a5918073e9c402a8b642c5897dd8e87/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:10.166882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 901/1806] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:10.178705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:10.179312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:10.181121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:10.318450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] MFA Rules not processed for user `0a5918073e9c402a8b642c5897dd8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:10.325510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:10.466660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 906/1807] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:10.472201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:10.493344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77f93-e628-423a-92b3-20c3dcfab7b2 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:10.608252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:10.690969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857cb9b-f21e-43e2-90ea-08ed8e8ba560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:10.693912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857cb9b-f21e-43e2-90ea-08ed8e8ba560 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:10.694798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857cb9b-f21e-43e2-90ea-08ed8e8ba560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:10.695197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857cb9b-f21e-43e2-90ea-08ed8e8ba560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:10.699159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857cb9b-f21e-43e2-90ea-08ed8e8ba560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:10.711199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857cb9b-f21e-43e2-90ea-08ed8e8ba560 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:10.774289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 902/1808] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:10 2022] POST /identity/v3/projects => generated 375 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:10.779141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:10.804640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cbf6d-f431-47ba-b685-1dcc2b181c9b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:10.911095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:10.911748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:10.912198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:10.948941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] MFA Rules not processed for user `c18c8d4034ce40dba9f9f48e96a19814`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:10.951060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:11.009971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:11.010697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:11.011139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:11.011439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:11.012154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:11.014162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:11.056533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 907/1809] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:10 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:11.064659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:11.096966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be518f5-390f-430f-9395-68ad79d1c7d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:11.097162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 903/1810] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:11.111167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:11.114404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:11.117986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:11.118461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:11.119413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:11.129306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:11.162081 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:11.180294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:11.182201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 908/1811] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:11.225129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f406114-fb0a-4d9e-b762-587fc1a312a5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:11.243169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:11.247101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:11.247915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:11.250005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] PATH_INFO: `/v3/projects/30b91619e8b343a8b6513e5c33cf42f1/users/06c7275084444919b6e2d5d6b5937772/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:11.325593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b91619e8b343a8b6513e5c33cf42f1, user_id=06c7275084444919b6e2d5d6b5937772, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:11.330422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:11.380273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 904/1812] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:11 2022] PUT /identity/v3/projects/30b91619e8b343a8b6513e5c33cf42f1/users/06c7275084444919b6e2d5d6b5937772/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:11.385636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:11.402294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88d0e-488a-4c7f-998b-27bb13b9ac6a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:11.613683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909987d2-c76c-43fa-9326-a354527fb552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:11.614389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909987d2-c76c-43fa-9326-a354527fb552 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:11.614676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909987d2-c76c-43fa-9326-a354527fb552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:11.614955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909987d2-c76c-43fa-9326-a354527fb552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:11.621291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909987d2-c76c-43fa-9326-a354527fb552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:11.623452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909987d2-c76c-43fa-9326-a354527fb552 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:11.686857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 909/1813] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:11.691347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:11.755398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afeb5b-0ed2-41d6-8b61-b64f3bf4e259 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:11.786167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:11.795941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:11.795941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:11.795941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] PATH_INFO: `/v3/projects/30b91619e8b343a8b6513e5c33cf42f1/users/06c7275084444919b6e2d5d6b5937772/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:11.818864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b91619e8b343a8b6513e5c33cf42f1, user_id=06c7275084444919b6e2d5d6b5937772, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:11.823840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:11.886489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 905/1814] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:11 2022] PUT /identity/v3/projects/30b91619e8b343a8b6513e5c33cf42f1/users/06c7275084444919b6e2d5d6b5937772/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:11.900335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0f9f6-8e26-4059-aec7-7d45649e815c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:11.900801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0f9f6-8e26-4059-aec7-7d45649e815c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:11.901423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0f9f6-8e26-4059-aec7-7d45649e815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:11.983141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b6b0f9f6-8e26-4059-aec7-7d45649e815c None None] MFA Rules not processed for user `06c7275084444919b6e2d5d6b5937772`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:11.984561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b6b0f9f6-8e26-4059-aec7-7d45649e815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:12.261700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 910/1815] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:12.275215 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:12.282269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:12.423587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bd473-c17a-474e-b528-634b83836a9b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:12.431644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:12.434449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:12.434449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:12.434822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:12.507277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:12.550221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:12.552269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:12.590338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 906/1816] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:12.952378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:12.952675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:12.953040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:13.110352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:13.112921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:13.316467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:13.347191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 911/1817] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:28:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:13.574249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:13.576417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:13.906460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed23e1a-6240-450d-b9ca-b468d27d31ed service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:13.922709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a842c7-30a6-4374-94bc-f9d5e02bc1ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:13.924233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a842c7-30a6-4374-94bc-f9d5e02bc1ce service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:13.925297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a842c7-30a6-4374-94bc-f9d5e02bc1ce service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:13.926183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a842c7-30a6-4374-94bc-f9d5e02bc1ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:13.951644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a842c7-30a6-4374-94bc-f9d5e02bc1ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:13.953920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a842c7-30a6-4374-94bc-f9d5e02bc1ce service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:13.968832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 907/1818] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:28:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:28:15.083319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:15.084988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:15.283257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b8b63-d016-4bb1-b32f-a0d51a94103c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:15.296415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:15.297630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:15.298070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:15.298498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] PATH_INFO: `/v3/users/a0469e93b5504705b1c4ea0641f4499b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:15.302663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0469e93b5504705b1c4ea0641f4499b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:15.305581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:15.488913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0469e93b5504705b1c4ea0641f4499b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:15.514082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0469e93b5504705b1c4ea0641f4499b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:15.575687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0469e93b5504705b1c4ea0641f4499b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:15.576732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0469e93b5504705b1c4ea0641f4499b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:15.609224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0469e93b5504705b1c4ea0641f4499b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:15.672472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 912/1819] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:15 2022] DELETE /identity/v3/users/a0469e93b5504705b1c4ea0641f4499b => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:15.673676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:15.674174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:15.674380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:15.736122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:15.740263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:15.994535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 908/1820] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:16.039473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:16.042303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.148072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:16.149748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.160657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce9cbf-b4af-405b-ac63-d197181c231e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:16.169993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:16.170665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.170949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.171314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.187223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:16.188707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:16.200093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 913/1821] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:16.233593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:16.248678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0618f-a482-4266-ab3d-4e80d7236ed8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:16.257134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:16.258387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.258822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.259743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.262767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.364065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:16.366061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708831-b36c-4d0f-8d3b-0e6a00acab00 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:16.384126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 909/1822] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:16.611747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:16.613661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.637387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a146ec-8f00-46f8-90dc-9170f16d0a45 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:16.666243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:16.668453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.669272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.670131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] PATH_INFO: `/v3/projects/b28889e9b7f44903b09e87c964765d4b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.674867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28889e9b7f44903b09e87c964765d4b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:16.677291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:16.719118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:16.806982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28889e9b7f44903b09e87c964765d4b', 'request_id': 'req-2f4827c8-acf9-4dad-b652-67eb8beea238'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.813978 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28889e9b7f44903b09e87c964765d4b', 'request_id': 'req-2f4827c8-acf9-4dad-b652-67eb8beea238'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.825812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28889e9b7f44903b09e87c964765d4b', 'request_id': 'req-2f4827c8-acf9-4dad-b652-67eb8beea238'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.857754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] The token cache is being invalidate because project b28889e9b7f44903b09e87c964765d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:16.858172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.858949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2f4827c8-acf9-4dad-b652-67eb8beea238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.865236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 914/1823] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:16 2022] DELETE /identity/v3/projects/b28889e9b7f44903b09e87c964765d4b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:16.867640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.869166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.869887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.894350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:16.896101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.043999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.044568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.044773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.051762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 910/1824] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:17.076343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:17.081195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.114804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 915/1825] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:17.119349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.137796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8a193-ccd8-4f53-ac05-3a163403a486 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.157650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.160708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.164442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.164442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.170897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:17.174515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.176400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.206837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 911/1826] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:17.212122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.223143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f9f2-5e47-463b-bd80-055a7b03d01b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.230127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.230800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.231085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.231575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.232383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.234322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.268002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 916/1827] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:17.273415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.285267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33face7-8ffa-4aa6-aca1-dd6fccd17d34 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.361551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.362236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.362527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.362804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.363528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.365484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.385842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 912/1828] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:17.389988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.400914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cf9a7-16a3-484d-95c6-9d88546a387f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.405446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.406102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.406386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.406664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.407395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.409312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.419821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 917/1829] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:28:17.423824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.434053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262bc7e-e6fb-4f58-a3a0-098725f2295e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.438727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd743fb-4874-40de-b708-103016c8cb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.439487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd743fb-4874-40de-b708-103016c8cb30 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.439845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd743fb-4874-40de-b708-103016c8cb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.440149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd743fb-4874-40de-b708-103016c8cb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.441198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd743fb-4874-40de-b708-103016c8cb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.443275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd743fb-4874-40de-b708-103016c8cb30 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.454519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 913/1830] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:17.458633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.469615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda1a01-0fab-4152-959e-031a8c67943e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.474150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.475025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.475311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.475682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] PATH_INFO: `/v3/projects/d59cb81c13924c599ae44d351268a960/users/396d8057322842a094fddc09079f5925/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.490764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59cb81c13924c599ae44d351268a960, user_id=396d8057322842a094fddc09079f5925, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.493729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.506626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 918/1831] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:17 2022] PUT /identity/v3/projects/d59cb81c13924c599ae44d351268a960/users/396d8057322842a094fddc09079f5925/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:17.518436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.518728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.519007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.533475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:17.551584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] MFA Rules not processed for user `396d8057322842a094fddc09079f5925`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:17.552818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.628550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 914/1832] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:17.641125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.642884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.722712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d52c7-74af-47d2-af90-31bca2d89a35 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.726539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f13466-8a9c-4e14-8b6f-0a9712ffa566 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.727194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f13466-8a9c-4e14-8b6f-0a9712ffa566 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.727475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f13466-8a9c-4e14-8b6f-0a9712ffa566 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.727748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f13466-8a9c-4e14-8b6f-0a9712ffa566 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.738288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f13466-8a9c-4e14-8b6f-0a9712ffa566 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.739750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f13466-8a9c-4e14-8b6f-0a9712ffa566 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.750345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 919/1833] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:18.812938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:18.815243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:18.834618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:18.835487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:18.835804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:18.922672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:18.931365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:18.969072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1061b-8eec-434f-85c1-a4998e81aca8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:18.993927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:18.995588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:18.996094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:18.999718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:19.009671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:19.010548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:19.172966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 920/1834] 10.222.0.96 () {58 vars in 1015 bytes} [Wed Jul 20 19:28:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:19.225620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:19.227898 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:19.262253 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e409f-86cb-4d1b-a70f-bd07af6d33f4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:19.277834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:19.281275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:19.282197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761e8a1-bfae-4c2a-b09d-85bd85862155 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:19.283226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:19.283977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:19.284547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:19.306850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:19.313153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 915/1835] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:19.314678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:19.315677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:19.315972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:19.318079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:19.352394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 921/1836] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:19.357786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:19.360988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:19.373963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:19.383274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:19.396229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeeaf41-3bae-49d3-87a2-fb7e7988d3f7 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:19.406171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:19.409181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:19.410563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:19.411231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:19.443565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:19.448588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:19.488830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 922/1837] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:19.536790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:19.585519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 916/1838] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:28:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:19.605946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:19.609509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:19.840153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6932-e141-4755-ba9c-987f4959b0fc service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:19.856859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8b12-9ac2-406c-91b1-d907b1e4e96b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:19.857711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8b12-9ac2-406c-91b1-d907b1e4e96b service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:19.858096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8b12-9ac2-406c-91b1-d907b1e4e96b service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:19.858482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8b12-9ac2-406c-91b1-d907b1e4e96b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:19.879423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8b12-9ac2-406c-91b1-d907b1e4e96b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:19.881301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8b12-9ac2-406c-91b1-d907b1e4e96b service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:19.904804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 923/1839] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:28:21.668933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:21.670422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:21.673216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6b10d-f959-4929-b01a-f16014c3833d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:21.673721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6b10d-f959-4929-b01a-f16014c3833d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:21.673817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6b10d-f959-4929-b01a-f16014c3833d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:21.676459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 924/1840] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:28:21.716384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:21.716673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:21.717151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:21.756568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:21.758430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:21.807923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24c64d-0828-4895-9f3d-8b00f9f2c5b7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:21.814975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:21.815893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:21.816218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:21.816500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] PATH_INFO: `/v3/users/7bb70e74e54940ec9f29e9f656fe721e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:21.818651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb70e74e54940ec9f29e9f656fe721e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:21.822699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:21.905316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 925/1841] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:28:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:21.906706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb70e74e54940ec9f29e9f656fe721e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:21.916258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb70e74e54940ec9f29e9f656fe721e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:21.925666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:21.927797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:21.936986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb70e74e54940ec9f29e9f656fe721e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:21.937789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb70e74e54940ec9f29e9f656fe721e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:21.947863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2db7fa-6713-45bd-9a77-794d471daa0a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:21.948152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-04f0117d-e06a-4d06-9e62-e2a9812bc4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb70e74e54940ec9f29e9f656fe721e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:21.956032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:21.956711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:21.957037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:21.957332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:21.959182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:21.962324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 917/1842] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:21 2022] DELETE /identity/v3/users/7bb70e74e54940ec9f29e9f656fe721e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:21.963869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cebed8-b743-42d0-bb7f-663e1c672663 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:21.964235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cebed8-b743-42d0-bb7f-663e1c672663 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:21.964590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cebed8-b743-42d0-bb7f-663e1c672663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:21.985691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:21.987999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.005310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-95cebed8-b743-42d0-bb7f-663e1c672663 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:22.006650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95cebed8-b743-42d0-bb7f-663e1c672663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:22.009989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 926/1843] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:22.098827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 918/1844] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:22.108486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.113644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:22.133741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9092-e980-4a60-a071-868b727558e6 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.141146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:22.196574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4443e-2b66-4651-8c7d-11a4b160b6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.197386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4443e-2b66-4651-8c7d-11a4b160b6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.197744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4443e-2b66-4651-8c7d-11a4b160b6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.198115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4443e-2b66-4651-8c7d-11a4b160b6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.208009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4443e-2b66-4651-8c7d-11a4b160b6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.209598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4443e-2b66-4651-8c7d-11a4b160b6d9 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.226210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 927/1845] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:22.309718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b4d0-3ada-4398-b613-d274fd6934ff None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.310044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b4d0-3ada-4398-b613-d274fd6934ff None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.310980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b4d0-3ada-4398-b613-d274fd6934ff None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.315525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 919/1846] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:28:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:28:22.322999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.323304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.323594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.357448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:22.359673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:22.452304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 928/1847] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:22.466107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682b4d0-3ada-4398-b613-d274fd6934ff None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.468164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e682b4d0-3ada-4398-b613-d274fd6934ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:22.487700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682b4d0-3ada-4398-b613-d274fd6934ff None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.494953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.495684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.495935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.496210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.505949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.507618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.513889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:22.521171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 920/1848] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:22.568890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.571610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:22.595329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cf820-83d5-44dc-b4d3-d8d4c06e6bd4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.610874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.611961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.612313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.612657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] PATH_INFO: `/v3/projects/b45df66c0ffd4dda8c342be251a91565` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.613971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b45df66c0ffd4dda8c342be251a91565)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.615948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.667611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b45df66c0ffd4dda8c342be251a91565', 'request_id': 'req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.674432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b45df66c0ffd4dda8c342be251a91565', 'request_id': 'req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.682463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b45df66c0ffd4dda8c342be251a91565', 'request_id': 'req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.697269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] The token cache is being invalidate because project b45df66c0ffd4dda8c342be251a91565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:22.697794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.698628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.700693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 929/1849] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:22 2022] DELETE /identity/v3/projects/b45df66c0ffd4dda8c342be251a91565 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:22.705666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.717944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd0fed-d594-4aeb-8275-6f81da796b9d service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.776163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.777082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.777395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.777672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] PATH_INFO: `/v3/users/73aeaaeb57734ea08ca1074efe11b643` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.778867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73aeaaeb57734ea08ca1074efe11b643)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.780786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.831378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73aeaaeb57734ea08ca1074efe11b643'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.839035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73aeaaeb57734ea08ca1074efe11b643'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.853185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73aeaaeb57734ea08ca1074efe11b643'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.853913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73aeaaeb57734ea08ca1074efe11b643'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.862107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73aeaaeb57734ea08ca1074efe11b643'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:22.868778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 921/1850] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:22 2022] DELETE /identity/v3/users/73aeaaeb57734ea08ca1074efe11b643 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:23.165890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:23.167914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:23.189715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23de5-6477-4ac8-a9c0-fea5e25a6c1c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:23.225384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:23.265354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:23.266217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:23.266501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:23.266777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] PATH_INFO: `/v3/projects/637ef0b4b7dd4c72a3ba3d42df740fe4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:23.267957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=637ef0b4b7dd4c72a3ba3d42df740fe4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:23.269929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:23.316443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637ef0b4b7dd4c72a3ba3d42df740fe4', 'request_id': 'req-705d571b-980f-4229-9742-9e8ba7800976'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.323153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637ef0b4b7dd4c72a3ba3d42df740fe4', 'request_id': 'req-705d571b-980f-4229-9742-9e8ba7800976'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.330737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637ef0b4b7dd4c72a3ba3d42df740fe4', 'request_id': 'req-705d571b-980f-4229-9742-9e8ba7800976'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.343717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] The token cache is being invalidate because project 637ef0b4b7dd4c72a3ba3d42df740fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:23.344012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.344683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.346261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 930/1851] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:23 2022] DELETE /identity/v3/projects/637ef0b4b7dd4c72a3ba3d42df740fe4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:23.350977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:23.352690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:23.366668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd15c7-5e2c-4a63-94b9-365182d2e288 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:23.426715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:23.427590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:23.427870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:23.428143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] PATH_INFO: `/v3/users/25d284b36ddc448fb582f6a74568b5e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:23.429956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d284b36ddc448fb582f6a74568b5e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:23.431910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:23.456012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:23.484853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d284b36ddc448fb582f6a74568b5e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.492835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d284b36ddc448fb582f6a74568b5e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.505594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d284b36ddc448fb582f6a74568b5e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.506322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d284b36ddc448fb582f6a74568b5e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.514640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-732d7aa3-491a-4dad-ac43-8337ebec0d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d284b36ddc448fb582f6a74568b5e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:23.521176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 922/1852] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:23 2022] DELETE /identity/v3/users/25d284b36ddc448fb582f6a74568b5e3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:23.963311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:23.964871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:23.987655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d571b-980f-4229-9742-9e8ba7800976 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.102054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.105617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.106104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.107293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] PATH_INFO: `/v3/projects/bbc172c2cbb94b3e8fe205cbee41b1f8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.110850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc172c2cbb94b3e8fe205cbee41b1f8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.114820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.205230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc172c2cbb94b3e8fe205cbee41b1f8', 'request_id': 'req-60aec097-d7a5-4e64-b811-c925029164dd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:24.213309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc172c2cbb94b3e8fe205cbee41b1f8', 'request_id': 'req-60aec097-d7a5-4e64-b811-c925029164dd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:24.215504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:24.224937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc172c2cbb94b3e8fe205cbee41b1f8', 'request_id': 'req-60aec097-d7a5-4e64-b811-c925029164dd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:24.249096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] The token cache is being invalidate because project bbc172c2cbb94b3e8fe205cbee41b1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:24.249589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:24.250348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-60aec097-d7a5-4e64-b811-c925029164dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:24.252195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 931/1853] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:23 2022] DELETE /identity/v3/projects/bbc172c2cbb94b3e8fe205cbee41b1f8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:24.259970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.260437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.260534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.300747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:24.306938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:24.410432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 923/1854] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:24.420019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.420311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.420590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.440251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:24.441603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:24.458237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 932/1855] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:24.459121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:24.469918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f90f2-d3d6-49dd-bce7-3de4ea8424ac None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.474776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.475645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.476041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.476267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.477051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:24.477408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.479028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.492479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 924/1856] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:28:24.496704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:24.506362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1c13e-31cf-4730-bfd5-4f4064bd0fca None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.511754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.512359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.512677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.512992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.513695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.515602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.549029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 933/1857] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:24 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:24.553219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:24.564171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac4a1e-5acb-4401-a6d6-48a4ebf91286 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.629645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.630686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.631050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.631520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.633480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.635494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.672914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 925/1858] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:24 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:24.676620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:24.703590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c94e67-d3dc-4036-806a-91b86d078dd9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.710489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.711288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.711711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.712108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.713125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.714893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.738130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 934/1859] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:24.739292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:24.750422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1776f-19eb-4239-ba32-674822f5b2b0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.757736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886f7ab-8951-4906-bc46-f3eb1ccf86a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.758452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886f7ab-8951-4906-bc46-f3eb1ccf86a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.759399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886f7ab-8951-4906-bc46-f3eb1ccf86a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.759695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886f7ab-8951-4906-bc46-f3eb1ccf86a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.760444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886f7ab-8951-4906-bc46-f3eb1ccf86a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.762152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886f7ab-8951-4906-bc46-f3eb1ccf86a2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.781403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 926/1860] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:24.785922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:24.808050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f2c4-48cf-4b6a-9248-9e1f17b5363f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:24.829723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:24.830722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.831022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.831332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] PATH_INFO: `/v3/projects/2933a2d465444fff9287314059e53df8/users/855e62708e4045dc8d2b8b946fd3cd4b/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.870438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2933a2d465444fff9287314059e53df8, user_id=855e62708e4045dc8d2b8b946fd3cd4b, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:24.875549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:24.894239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 935/1861] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:24 2022] PUT /identity/v3/projects/2933a2d465444fff9287314059e53df8/users/855e62708e4045dc8d2b8b946fd3cd4b/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:24.906998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:24.907290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:24.907568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:24.915112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:24.938720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] MFA Rules not processed for user `855e62708e4045dc8d2b8b946fd3cd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:24.939975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:25.006338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 927/1862] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:25.011072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.081715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91190c99-039e-4c61-a19a-660a5e625265 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.086938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.087635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.087947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.088229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.088983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.090911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.125114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 936/1863] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:25 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:25.132583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.146584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc920f-0d7c-479c-8f36-a6a6cb239e00 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.237614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.238421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.238846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.239271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.240070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.242768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.268676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 928/1864] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:25 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:25.269696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.284667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f44ab-a243-48a1-b2a4-64480882a80f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.288934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.289749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.290043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.290325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.291295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.293004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.306284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 937/1865] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:28:25.310126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.328398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6941898-aba2-465d-aa61-42fa1e34b12d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.333460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.334371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.335228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.335514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] PATH_INFO: `/v3/projects/55141660f0f14e79b9badfeec1983dba/users/3086e73e8cee448a88c46f6d48cdc877/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.357118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55141660f0f14e79b9badfeec1983dba, user_id=3086e73e8cee448a88c46f6d48cdc877, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.359793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.381451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 929/1866] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:25 2022] PUT /identity/v3/projects/55141660f0f14e79b9badfeec1983dba/users/3086e73e8cee448a88c46f6d48cdc877/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:25.385638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.402131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955c11-ac25-4028-af09-e308f4932018 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.491385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:25.501859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2396ed3-f025-46c3-a666-6500c7ebb8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.502850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2396ed3-f025-46c3-a666-6500c7ebb8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.503297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2396ed3-f025-46c3-a666-6500c7ebb8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.503973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2396ed3-f025-46c3-a666-6500c7ebb8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.505479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2396ed3-f025-46c3-a666-6500c7ebb8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.507994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2396ed3-f025-46c3-a666-6500c7ebb8c8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.535997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 938/1867] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:25.537320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.551438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570262f4-7376-44b0-a3d2-e939e9f71da4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.557480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.558462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.558813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.559157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] PATH_INFO: `/v3/projects/55141660f0f14e79b9badfeec1983dba/users/3086e73e8cee448a88c46f6d48cdc877/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.561892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] RBAC: Authorizing `identity:create_grant(project_id=55141660f0f14e79b9badfeec1983dba, user_id=3086e73e8cee448a88c46f6d48cdc877, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.569463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.579854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 930/1868] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:25 2022] PUT /identity/v3/projects/55141660f0f14e79b9badfeec1983dba/users/3086e73e8cee448a88c46f6d48cdc877/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:25.592567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.593264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.593647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.626725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] MFA Rules not processed for user `3086e73e8cee448a88c46f6d48cdc877`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:25.628112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:25.725504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 939/1869] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:25.737859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.740098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:25.798223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:25.821782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb9d09-8d45-4ffb-8924-557d2880daba admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.826363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.827320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.827786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.828209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:25.839368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:25.841343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:25.855668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 931/1870] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:25.971745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:25.973185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:25.985004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e17f67-b340-4927-8e1c-e861fc111398 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:25.988598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7fd79-d2b9-40a6-ad13-ffc1b3d1c342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:25.989283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7fd79-d2b9-40a6-ad13-ffc1b3d1c342 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:25.989580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7fd79-d2b9-40a6-ad13-ffc1b3d1c342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:25.989855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7fd79-d2b9-40a6-ad13-ffc1b3d1c342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:26.054016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7fd79-d2b9-40a6-ad13-ffc1b3d1c342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:26.056487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7fd79-d2b9-40a6-ad13-ffc1b3d1c342 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:26.067036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 940/1871] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:35.847141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:36.061964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=65 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:36.619471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:36.722420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adb4c5-e878-4794-9866-e2c1cf86eb3b service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:36.731839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:36.733411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:36.733411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:36.733825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] PATH_INFO: `/v3/users/855e62708e4045dc8d2b8b946fd3cd4b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:36.762470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=855e62708e4045dc8d2b8b946fd3cd4b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:36.767451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:36.842270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '855e62708e4045dc8d2b8b946fd3cd4b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:36.852289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '855e62708e4045dc8d2b8b946fd3cd4b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:36.868128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '855e62708e4045dc8d2b8b946fd3cd4b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:36.868871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '855e62708e4045dc8d2b8b946fd3cd4b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:36.883356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '855e62708e4045dc8d2b8b946fd3cd4b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:36.898657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 932/1872] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:36 2022] DELETE /identity/v3/users/855e62708e4045dc8d2b8b946fd3cd4b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:36.901919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:36.902223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:36.902513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:36.949991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:36.953025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:37.097159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 941/1873] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:37.156207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:37.160131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:37.431473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f7507-8d7f-4932-af43-29a6f79e5dda admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:37.444927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:37.445210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:37.445446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:37.446761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:37.482948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:37.485162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:37.505331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 933/1874] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:38.021453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:38.023862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:38.051016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6a3a5-b2a0-4cac-9d93-27f557e5b5d2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:38.061407 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:38.062569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.062857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.063136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] PATH_INFO: `/v3/projects/2933a2d465444fff9287314059e53df8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.064348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2933a2d465444fff9287314059e53df8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:38.066322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:38.124034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2933a2d465444fff9287314059e53df8', 'request_id': 'req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.137972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2933a2d465444fff9287314059e53df8', 'request_id': 'req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.147940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2933a2d465444fff9287314059e53df8', 'request_id': 'req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.160394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] The token cache is being invalidate because project 2933a2d465444fff9287314059e53df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:38.160771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.161968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.164663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 942/1875] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:38 2022] DELETE /identity/v3/projects/2933a2d465444fff9287314059e53df8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:38.168711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:38.179549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888e71-25e4-408e-8022-dfe31ffff752 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:38.198463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:38.239568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:38.240450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.240781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.241106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] PATH_INFO: `/v3/users/3086e73e8cee448a88c46f6d48cdc877` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.242292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3086e73e8cee448a88c46f6d48cdc877)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:38.244288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:38.296178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3086e73e8cee448a88c46f6d48cdc877'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.303047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3086e73e8cee448a88c46f6d48cdc877'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.314930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3086e73e8cee448a88c46f6d48cdc877'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.315631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3086e73e8cee448a88c46f6d48cdc877'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.332467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ad8aa1e9-ffa4-4720-a9c3-9f244d279fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3086e73e8cee448a88c46f6d48cdc877'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.340416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 934/1876] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:38 2022] DELETE /identity/v3/users/3086e73e8cee448a88c46f6d48cdc877 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:38.629158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:38.630779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:38.645853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbf3cc-ee30-4212-a69a-ecd7405bb6bc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:38.706224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:38.707090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.707394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.707666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] PATH_INFO: `/v3/projects/55141660f0f14e79b9badfeec1983dba` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.708835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55141660f0f14e79b9badfeec1983dba)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:38.711619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:38.746969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:38.767522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55141660f0f14e79b9badfeec1983dba', 'request_id': 'req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.773209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55141660f0f14e79b9badfeec1983dba', 'request_id': 'req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.782379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55141660f0f14e79b9badfeec1983dba', 'request_id': 'req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.793960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] The token cache is being invalidate because project 55141660f0f14e79b9badfeec1983dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:38.794251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.794902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5830fc8e-0ec4-4141-af5f-d2c3c247ff63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:38.796508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 943/1877] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:38 2022] DELETE /identity/v3/projects/55141660f0f14e79b9badfeec1983dba => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:38.801545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.801840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.802123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.821030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:38.822267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:38.889396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 935/1878] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:38.897452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.897728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.898070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.916742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:38.918624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:38.926729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 944/1879] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:38.931705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:38.941486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6a2c8-d985-42b2-bb1c-b39f18087542 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:38.946974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:38.947647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.947966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.948245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.949008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:38.949345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:38.950952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:38.966765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 936/1880] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:38.968353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:38.978690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2926a33-1a7c-407a-b53c-a165f711ecbe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:38.983261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:38.983920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:38.984207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:38.984481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:38.985224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:38.987115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.015713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 945/1881] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:38 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:39.020124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.030207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59b3d6-52ae-442e-be0f-ea0d62d81a1b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.057476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:39.096380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.096996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.097286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.097564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.098245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.100708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.121063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 937/1882] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:39 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:39.125365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.136154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d829b-7f1b-4483-b5e3-a7ed2952ca16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.140765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.141459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.141745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.142020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.142749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.144373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.160622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 946/1883] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:39.161390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.173380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fbd92-82f5-4c5c-96d4-1fe7ebc78805 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.178944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881501a-628a-4172-8dfd-7ec809cf50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.180074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881501a-628a-4172-8dfd-7ec809cf50d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.180578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881501a-628a-4172-8dfd-7ec809cf50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.181140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881501a-628a-4172-8dfd-7ec809cf50d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.182493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881501a-628a-4172-8dfd-7ec809cf50d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.185552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881501a-628a-4172-8dfd-7ec809cf50d4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.196861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 938/1884] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:28:39.201437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.214544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97e575-29d6-483d-bc67-8b9d7b892194 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.219608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.221266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.221628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.221992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] PATH_INFO: `/v3/projects/819ea25504bf403a9c929b179f2c4637/users/af3606b7d2404d53b4fba856f823a1a2/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.239664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819ea25504bf403a9c929b179f2c4637, user_id=af3606b7d2404d53b4fba856f823a1a2, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.243211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.260273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 947/1885] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:39 2022] PUT /identity/v3/projects/819ea25504bf403a9c929b179f2c4637/users/af3606b7d2404d53b4fba856f823a1a2/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:39.272289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.272727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.273104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.317288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] MFA Rules not processed for user `af3606b7d2404d53b4fba856f823a1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:39.318740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:39.418315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 939/1886] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:39.427551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.430105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:39.500813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155d2be-b6aa-4af8-b3f1-790b6311d4c7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.504533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6ff29-15a1-4488-bd13-7f0695403873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.505256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6ff29-15a1-4488-bd13-7f0695403873 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.505558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6ff29-15a1-4488-bd13-7f0695403873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.505832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6ff29-15a1-4488-bd13-7f0695403873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.516097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6ff29-15a1-4488-bd13-7f0695403873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.517583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6ff29-15a1-4488-bd13-7f0695403873 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.530493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 948/1887] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:39.555024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.628787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03b331-a5d8-42e1-aa76-eed7ee782e39 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.636940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.637864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.638161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.638409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] PATH_INFO: `/v3/users/af3606b7d2404d53b4fba856f823a1a2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.639666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3606b7d2404d53b4fba856f823a1a2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.641656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.691536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:39.699199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3606b7d2404d53b4fba856f823a1a2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:39.708441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3606b7d2404d53b4fba856f823a1a2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:39.725872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3606b7d2404d53b4fba856f823a1a2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:39.727216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3606b7d2404d53b4fba856f823a1a2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:39.738180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3606b7d2404d53b4fba856f823a1a2'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:39.745996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 940/1888] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:39 2022] DELETE /identity/v3/users/af3606b7d2404d53b4fba856f823a1a2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:39.751177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.751470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.751755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.770201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:39.771566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:39.831557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:39.846747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 949/1889] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:39.861783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:39.863001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:39.939817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706e738-7c0d-41b0-97f1-0520785c4f0e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:39.947430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3fd72-4873-4897-9c1c-8f21b1c8481d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:39.948286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3fd72-4873-4897-9c1c-8f21b1c8481d service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:39.948802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3fd72-4873-4897-9c1c-8f21b1c8481d service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:39.949436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3fd72-4873-4897-9c1c-8f21b1c8481d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:39.962623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3fd72-4873-4897-9c1c-8f21b1c8481d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:39.964091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3fd72-4873-4897-9c1c-8f21b1c8481d service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:39.977027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 941/1890] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:40.298646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.299687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:40.313533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298902a-2b9b-417a-b9b9-f14515121578 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.318546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.320670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.321012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.321328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] PATH_INFO: `/v3/projects/819ea25504bf403a9c929b179f2c4637` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.322547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819ea25504bf403a9c929b179f2c4637)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.324507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.376258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819ea25504bf403a9c929b179f2c4637', 'request_id': 'req-92d303b9-216b-4d4f-a659-4838a1482258'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:40.382854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819ea25504bf403a9c929b179f2c4637', 'request_id': 'req-92d303b9-216b-4d4f-a659-4838a1482258'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:40.392294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819ea25504bf403a9c929b179f2c4637', 'request_id': 'req-92d303b9-216b-4d4f-a659-4838a1482258'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:40.405843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] The token cache is being invalidate because project 819ea25504bf403a9c929b179f2c4637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:40.406130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:40.406831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-92d303b9-216b-4d4f-a659-4838a1482258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:40.408524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 950/1891] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:40 2022] DELETE /identity/v3/projects/819ea25504bf403a9c929b179f2c4637 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:40.415512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.416032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.416210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.437751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:40.440502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:40.531213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 942/1892] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:40.541010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.541316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.541602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.563970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:40.565238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:40.573816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 951/1893] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:40.577840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.589178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e501f-3530-4fdc-b50e-e73a22b4f67e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.597745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.598932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.599418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.600036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.601907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:40.602557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.605730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.633994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 943/1894] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:40.639415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.650042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b43131-f4aa-49b4-9881-943f6eef8a3a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.655224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.655973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.656335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.656666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.657496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.660299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.722985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 952/1895] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:40 2022] POST /identity/v3/projects => generated 359 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:40.727230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.738075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e6788-560e-44d1-8a1e-319b2d8f7f00 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.795285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:40.809678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.810356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.810639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.810912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.811636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.813593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.837333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 944/1896] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:40 2022] POST /identity/v3/users => generated 280 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:28:40.838838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.849063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1aaabf-ad34-4213-95d8-147a52e8559d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.853976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.854632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.854915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.855190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.855913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.857560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.867926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 953/1897] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:40.871843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.882894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5220c5-e290-4f02-a49c-f3bac75caa08 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.887795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcd998-b4d9-4f58-b8b2-9d9e1cfbf28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.888485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcd998-b4d9-4f58-b8b2-9d9e1cfbf28e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.888775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcd998-b4d9-4f58-b8b2-9d9e1cfbf28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.889101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcd998-b4d9-4f58-b8b2-9d9e1cfbf28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.889842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcd998-b4d9-4f58-b8b2-9d9e1cfbf28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.891479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcd998-b4d9-4f58-b8b2-9d9e1cfbf28e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.906013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 945/1898] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:40.911718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.922777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed4f85-088d-4bb6-813a-b8a829f15090 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.927597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.928528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.928807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.929123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] PATH_INFO: `/v3/projects/0542a4825011418a92398b8a6f8bf62b/users/3fc53cbb0e574390b53c2703dce61ac5/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.948343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0542a4825011418a92398b8a6f8bf62b, user_id=3fc53cbb0e574390b53c2703dce61ac5, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.951602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:40.969086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 954/1899] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:40 2022] PUT /identity/v3/projects/0542a4825011418a92398b8a6f8bf62b/users/3fc53cbb0e574390b53c2703dce61ac5/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:40.979987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.980273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.980556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.009247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] MFA Rules not processed for user `3fc53cbb0e574390b53c2703dce61ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:41.010503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.089730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 946/1900] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:41.124633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.126414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.191594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60639b67-d064-4109-8a0d-684c94c14358 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.195256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989a22f-ac85-49b3-9511-83d18fca5493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.195916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989a22f-ac85-49b3-9511-83d18fca5493 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.196205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989a22f-ac85-49b3-9511-83d18fca5493 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.196483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989a22f-ac85-49b3-9511-83d18fca5493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.205497 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989a22f-ac85-49b3-9511-83d18fca5493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.207002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989a22f-ac85-49b3-9511-83d18fca5493 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.216636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 955/1901] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:41.232867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.306213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acdaae-cc3e-44ec-9d21-2e8f34d8e7d9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.310847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.311723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.312007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.312280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] PATH_INFO: `/v3/users/3fc53cbb0e574390b53c2703dce61ac5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.313512 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc53cbb0e574390b53c2703dce61ac5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.315421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.369560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc53cbb0e574390b53c2703dce61ac5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.378622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc53cbb0e574390b53c2703dce61ac5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.380584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:41.396399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc53cbb0e574390b53c2703dce61ac5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.398753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc53cbb0e574390b53c2703dce61ac5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.407588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc53cbb0e574390b53c2703dce61ac5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.414493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 947/1902] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:41 2022] DELETE /identity/v3/users/3fc53cbb0e574390b53c2703dce61ac5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:41.419670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.420258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.420258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.441228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:41.442584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.454607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:41.518320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 956/1903] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:41.527255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.528922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.612583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915bf46-6f51-4e85-88ee-4d652d5de82d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.616405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281161c-b35d-4b6f-8a7a-bbe6e77b0b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.617105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281161c-b35d-4b6f-8a7a-bbe6e77b0b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.617388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281161c-b35d-4b6f-8a7a-bbe6e77b0b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.617667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281161c-b35d-4b6f-8a7a-bbe6e77b0b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.627043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281161c-b35d-4b6f-8a7a-bbe6e77b0b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.628472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281161c-b35d-4b6f-8a7a-bbe6e77b0b29 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.639370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 948/1904] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:42.101689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.103154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.117921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5508785-7ba2-4f96-aceb-9b35a06a9d38 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.124565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.126199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.126650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.127066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] PATH_INFO: `/v3/projects/0542a4825011418a92398b8a6f8bf62b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.128943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0542a4825011418a92398b8a6f8bf62b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.130990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.205183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0542a4825011418a92398b8a6f8bf62b', 'request_id': 'req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:42.211592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0542a4825011418a92398b8a6f8bf62b', 'request_id': 'req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:42.219634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0542a4825011418a92398b8a6f8bf62b', 'request_id': 'req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:42.232985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] The token cache is being invalidate because project 0542a4825011418a92398b8a6f8bf62b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:42.233429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:42.234232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9d4266d3-1a99-46cd-8dd4-62ad592c31fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:42.236626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 957/1905] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:42 2022] DELETE /identity/v3/projects/0542a4825011418a92398b8a6f8bf62b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:42.243045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.243565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.243703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.263088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:42.264319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.329476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 949/1906] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:42.342151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.342574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.342958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.362029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:42.363235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.372349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 958/1907] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:42.376748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.386725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd552759-56f0-4e33-9fff-04c03b072079 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.391274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.391930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.392219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.392502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.393275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:42.393587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.395186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.407189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 950/1908] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:42.411630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.421660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4e0e2-5882-47c0-a81f-aee3737f133d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.426276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.426932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.427213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.427549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.428870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.430859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.457908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 959/1909] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:42 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:42.463751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.475904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104e4c-f95c-4b52-b9de-5eb0cfe27745 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.535880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.536604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.536901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.537213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.538634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.540868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.549767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:42.568276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 951/1910] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:42 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:28:42.570243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.580289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e0d08-a505-403a-aa17-8487622f96c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.599380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.600851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.601197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.601595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.602360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.604167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.618743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 960/1911] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:42.623329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.635015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d87a-580c-4da1-a89e-81f2b9210079 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.640376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a848a9b-acb4-4316-b232-7742e921b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.641741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848a9b-acb4-4316-b232-7742e921b19e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.642220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848a9b-acb4-4316-b232-7742e921b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.642669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a848a9b-acb4-4316-b232-7742e921b19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.643778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a848a9b-acb4-4316-b232-7742e921b19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.645859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a848a9b-acb4-4316-b232-7742e921b19e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.660577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 952/1912] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:42.664574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.674262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160ebc-96d2-450d-a418-e8d5660d08ef admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.679419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.680304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.680590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.680902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] PATH_INFO: `/v3/projects/3f1255754afb4cd384a11c87a9d717df/users/493d126879f44b26951920235e7f7730/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.701668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1255754afb4cd384a11c87a9d717df, user_id=493d126879f44b26951920235e7f7730, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.704793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.723427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 961/1913] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:42 2022] PUT /identity/v3/projects/3f1255754afb4cd384a11c87a9d717df/users/493d126879f44b26951920235e7f7730/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:42.729506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc8e79-510e-47fb-80e9-49f91065fc28 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.729799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc8e79-510e-47fb-80e9-49f91065fc28 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.730096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc8e79-510e-47fb-80e9-49f91065fc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.751649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-83dc8e79-510e-47fb-80e9-49f91065fc28 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:42.753162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-83dc8e79-510e-47fb-80e9-49f91065fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.816467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 953/1914] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:42.828295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.830218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.882209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:42.905838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ecb33-fffc-4d6d-aa66-273e823f1174 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.909869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.910520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.910806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.911110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.921283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.922718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.933095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 962/1915] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:49.511339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:49.511847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:49.511959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:49.542660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] MFA Rules not processed for user `493d126879f44b26951920235e7f7730`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:49.544029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:49.647160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 954/1916] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:49.651578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:49.666441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffca160-2aed-4f7b-8e7c-88a95921b32e service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:49.671984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:49.672651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:49.672956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:49.673391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:49.674244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:49.676346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:49.707661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 963/1917] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:49 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:49.711558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:49.723581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934acf2-167c-425b-ba55-e9facc3b0968 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:49.785401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:49.786056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:49.786336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:49.786611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:49.787814 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:49.789894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:49.814601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 955/1918] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:49 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:49.815748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:49.827235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1ed29-a927-4675-b0eb-8a1bd1b433a8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:49.831942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:49.832631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:49.832945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:49.833259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:49.834021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:49.836082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:49.852777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 964/1919] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:28:49.854043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:49.859314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:49.867976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31ff44-7df7-4ad8-9047-4d7f9fc4536d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:49.874824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:49.875842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:49.876126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:49.876400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] PATH_INFO: `/v3/projects/350f45adac63449687f2f244dc3bf3ce/users/fb82ee83de5d44b6b979dfebd04ccd29/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:49.893487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350f45adac63449687f2f244dc3bf3ce, user_id=fb82ee83de5d44b6b979dfebd04ccd29, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:49.896185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:49.911151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 956/1920] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:49 2022] PUT /identity/v3/projects/350f45adac63449687f2f244dc3bf3ce/users/fb82ee83de5d44b6b979dfebd04ccd29/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:49.916208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:49.927687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172aa0-9c77-465c-9bbf-3057f2eebfb4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:49.990062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:49.991161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:49.991615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:49.991900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:49.992629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:49.994279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:50.004328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 965/1921] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:28:50.008959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:50.019269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae8576-eea2-4c26-ac08-8ebe0c99e437 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:50.024832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323177a8-517b-43a3-9838-a3f58395e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:50.025758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323177a8-517b-43a3-9838-a3f58395e1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:50.026042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323177a8-517b-43a3-9838-a3f58395e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:50.026318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323177a8-517b-43a3-9838-a3f58395e1d9 admin admin] PATH_INFO: `/v3/projects/350f45adac63449687f2f244dc3bf3ce/users/fb82ee83de5d44b6b979dfebd04ccd29/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:50.028327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323177a8-517b-43a3-9838-a3f58395e1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350f45adac63449687f2f244dc3bf3ce, user_id=fb82ee83de5d44b6b979dfebd04ccd29, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:50.031148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323177a8-517b-43a3-9838-a3f58395e1d9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:50.043947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 957/1922] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:50 2022] PUT /identity/v3/projects/350f45adac63449687f2f244dc3bf3ce/users/fb82ee83de5d44b6b979dfebd04ccd29/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:53.329433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:53.331332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:53.367012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4aff4-8982-4d3c-ae46-cb718f6478a2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:53.627721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:53.628780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:53.629137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:53.629420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] PATH_INFO: `/v3/users/0a5918073e9c402a8b642c5897dd8e87` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:53.633798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5918073e9c402a8b642c5897dd8e87)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:53.633798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:53.769575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:53.781385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5918073e9c402a8b642c5897dd8e87'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:53.798286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5918073e9c402a8b642c5897dd8e87'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:53.836319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5918073e9c402a8b642c5897dd8e87'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:53.837591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5918073e9c402a8b642c5897dd8e87'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:53.878051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5918073e9c402a8b642c5897dd8e87'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:53.897797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 966/1923] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:53 2022] DELETE /identity/v3/users/0a5918073e9c402a8b642c5897dd8e87 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:53.903230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4bf2a-5b55-4b4f-a8c1-8a251c1304ab None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:53.903841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4bf2a-5b55-4b4f-a8c1-8a251c1304ab None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:53.903922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4bf2a-5b55-4b4f-a8c1-8a251c1304ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:53.972308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0ce4bf2a-5b55-4b4f-a8c1-8a251c1304ab None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:53.976421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0ce4bf2a-5b55-4b4f-a8c1-8a251c1304ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.128408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 958/1924] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:54.144763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:54.146315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.292258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d1e92-7466-4763-b50d-0a5980107c39 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:54.302144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:54.303165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:54.303503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:54.303787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:54.339736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b8e5-5f36-4696-ba9e-b0c82f508ca4 None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:54.340086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b8e5-5f36-4696-ba9e-b0c82f508ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:54.340309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4b8e5-5f36-4696-ba9e-b0c82f508ca4 None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:54.342115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 959/1925] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:28:54.361211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:54.363372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:54.375295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:54.375571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:54.375854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:54.413074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 967/1926] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:54.420761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:54.422135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.603629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 960/1927] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:28:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:54.795317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:54.797953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.835118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e707de-abe1-4993-a4b4-d5d96d8484ad service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:54.856493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1408cf-cfe4-425c-baea-b46d4ad6ce17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:54.861557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1408cf-cfe4-425c-baea-b46d4ad6ce17 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:54.861832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1408cf-cfe4-425c-baea-b46d4ad6ce17 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:54.861970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1408cf-cfe4-425c-baea-b46d4ad6ce17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:54.896056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1408cf-cfe4-425c-baea-b46d4ad6ce17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:54.898247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1408cf-cfe4-425c-baea-b46d4ad6ce17 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:54.961339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 968/1928] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:55.577511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:55.579629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:55.641126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ffae7-92b5-4da5-8340-dc546afd22b6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:55.674128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:55.679380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:55.679639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:55.679939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] PATH_INFO: `/v3/users/11ba25f4993645349c30131a4a51571c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:55.703045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ba25f4993645349c30131a4a51571c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:55.723406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:55.875922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ba25f4993645349c30131a4a51571c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:55.877195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:55.889197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ba25f4993645349c30131a4a51571c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:55.915203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ba25f4993645349c30131a4a51571c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:55.918695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ba25f4993645349c30131a4a51571c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:55.949490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ba25f4993645349c30131a4a51571c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:55.965006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 961/1929] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:55 2022] DELETE /identity/v3/users/11ba25f4993645349c30131a4a51571c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:55.966542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:55.967048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:55.967485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:56.034935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:56.037892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.186617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:56.189731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.331198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 969/1930] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:56.373396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:56.376030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.377236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d464-2d77-4917-bc19-fc482718027b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:56.386730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:56.387735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:56.388234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:56.388563 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] PATH_INFO: `/v3/projects/5b79c86cbfb14712a4b234130583f9f2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:56.390668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b79c86cbfb14712a4b234130583f9f2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:56.394047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:56.517472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b79c86cbfb14712a4b234130583f9f2', 'request_id': 'req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.551276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b79c86cbfb14712a4b234130583f9f2', 'request_id': 'req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.568504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b79c86cbfb14712a4b234130583f9f2', 'request_id': 'req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.590062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb9a5-4074-4479-b71c-a231b9ea22b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:56.591359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] The token cache is being invalidate because project 5b79c86cbfb14712a4b234130583f9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:56.591822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.593310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.596071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 962/1931] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:56 2022] DELETE /identity/v3/projects/5b79c86cbfb14712a4b234130583f9f2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:56.599778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:56.601866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.603528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:56.724316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806a3ac-a7ae-4cf1-b9cd-3ddff1b32300 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:56.738271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:56.739320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:56.739591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:56.739925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] PATH_INFO: `/v3/users/06c7275084444919b6e2d5d6b5937772` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:56.741317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c7275084444919b6e2d5d6b5937772)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:56.743290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:56.745284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:56.757372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:56.758776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:56.761064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:56.762176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:56.773208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:56.805073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:56.808463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:56.812424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c7275084444919b6e2d5d6b5937772'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.824610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c7275084444919b6e2d5d6b5937772'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.847169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 970/1932] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:56.859363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c7275084444919b6e2d5d6b5937772'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.862164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c7275084444919b6e2d5d6b5937772'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.875447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c7275084444919b6e2d5d6b5937772'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:56.883966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 963/1933] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:56 2022] DELETE /identity/v3/users/06c7275084444919b6e2d5d6b5937772 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:58.726856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:58.743221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09423e-8960-4ff5-b087-19a740b20947 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:58.837627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:58.840544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:58.921972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:58.923189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:58.923678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:58.924118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] PATH_INFO: `/v3/projects/30b91619e8b343a8b6513e5c33cf42f1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:58.926110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b91619e8b343a8b6513e5c33cf42f1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:58.929953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:58.978183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a25267-4e43-4397-820c-081e050235b5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:59.006464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:59.007379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.007707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.008169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] PATH_INFO: `/v3/projects/79c54327a19e4dd8b81c4dd5c0c4096c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.009666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c54327a19e4dd8b81c4dd5c0c4096c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:59.017626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:59.029218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b91619e8b343a8b6513e5c33cf42f1', 'request_id': 'req-218d810b-5f96-4335-9fb4-7dc2bcf1264c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.044946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b91619e8b343a8b6513e5c33cf42f1', 'request_id': 'req-218d810b-5f96-4335-9fb4-7dc2bcf1264c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.069471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b91619e8b343a8b6513e5c33cf42f1', 'request_id': 'req-218d810b-5f96-4335-9fb4-7dc2bcf1264c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.114062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] The token cache is being invalidate because project 30b91619e8b343a8b6513e5c33cf42f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:59.114367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.121136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-218d810b-5f96-4335-9fb4-7dc2bcf1264c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.135018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 971/1934] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:58 2022] DELETE /identity/v3/projects/30b91619e8b343a8b6513e5c33cf42f1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:59.138897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.141167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.141798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.195432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] MFA Rules not processed for user `fb82ee83de5d44b6b979dfebd04ccd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:59.197139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.222665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c54327a19e4dd8b81c4dd5c0c4096c', 'request_id': 'req-13f83d86-be3c-45e8-8fd5-1d16f318168a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.238096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c54327a19e4dd8b81c4dd5c0c4096c', 'request_id': 'req-13f83d86-be3c-45e8-8fd5-1d16f318168a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.244285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:59.257069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c54327a19e4dd8b81c4dd5c0c4096c', 'request_id': 'req-13f83d86-be3c-45e8-8fd5-1d16f318168a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.277916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] The token cache is being invalidate because project 79c54327a19e4dd8b81c4dd5c0c4096c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:59.278213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.279160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-13f83d86-be3c-45e8-8fd5-1d16f318168a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.284929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 964/1935] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:58 2022] DELETE /identity/v3/projects/79c54327a19e4dd8b81c4dd5c0c4096c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:59.286864 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13cf9-f31a-40d9-b213-7c3f5e1ae429 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.287310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13cf9-f31a-40d9-b213-7c3f5e1ae429 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.287734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13cf9-f31a-40d9-b213-7c3f5e1ae429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.316441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e4a13cf9-f31a-40d9-b213-7c3f5e1ae429 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:59.320778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e4a13cf9-f31a-40d9-b213-7c3f5e1ae429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.322254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 972/1936] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:59.323353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:59.325918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.346901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:59.447153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19ab2c-66fe-4ba9-9e17-ca2a8d204ae7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:59.454306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:59.455892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.457106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.460199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] PATH_INFO: `/v3/users/29c27b8cff0549af8e53504d6cbdabe5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.462066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29c27b8cff0549af8e53504d6cbdabe5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:59.467287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:59.471081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 965/1937] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:59.481012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.481832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.482405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.513383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:59.514690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.528053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 966/1938] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:59.535846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:59.556943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bd6ff-eb40-4f7a-aa65-a34e7ac71333 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:59.565018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29c27b8cff0549af8e53504d6cbdabe5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.569885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:59.570407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.570690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.570970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.572199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:59.572545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:59.574562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:59.583141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29c27b8cff0549af8e53504d6cbdabe5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.602474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 967/1939] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:59.603684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:59.606443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.607158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29c27b8cff0549af8e53504d6cbdabe5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.608173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29c27b8cff0549af8e53504d6cbdabe5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.620441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29c27b8cff0549af8e53504d6cbdabe5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:59.634185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 973/1940] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:59 2022] DELETE /identity/v3/users/29c27b8cff0549af8e53504d6cbdabe5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:59.636510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:59.649163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.780044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:59.781029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aac0e6-c99f-491b-8bf1-7f5107f5a2b3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:59.814675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f51f0-4b78-4e1b-9be3-44c4d22576d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:59.822632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:59.823577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.824019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.824424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.825317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:59.827345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:59.885248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 974/1941] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:59 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:59.889409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:59.905376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:59.907818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.908113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.908396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.910652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:59.966379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:59.986345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b2183-e25a-474d-b6b4-d09546f2d319 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:59.994600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:59.995653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:59.996214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:59.996501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:59.997574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.000055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.019720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.022673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.034264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 975/1942] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:59 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:00.036341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.052679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3ec78-b30a-463d-ac58-af860037a189 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.062056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.065278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.066872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.067283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.068100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.068686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 968/1943] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:00.075200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.105544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 976/1944] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:00.110195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.121817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.123654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ee7eb-b43a-4b51-8c42-3fc688e1b591 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.124219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.148378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845043c-e0d7-4319-9774-570aef3e3e53 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.151574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.152402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.152690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.155083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.156095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.158555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.158696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.159348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] PATH_INFO: `/v3/projects/e7b1549139e54022a4444bf2b22f27c4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.159674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.160992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b1549139e54022a4444bf2b22f27c4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.161639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.164380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.195862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 969/1945] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:00.197541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.215100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a8d02-c286-45de-887f-a414bee78db5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.221225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf7d54-3776-4d27-9969-d71393b9fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.222648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7d54-3776-4d27-9969-d71393b9fe86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.223500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7d54-3776-4d27-9969-d71393b9fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.223922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7d54-3776-4d27-9969-d71393b9fe86 admin admin] PATH_INFO: `/v3/projects/432db4bcef9d4d769509e7344a6c1942/users/43cade108cdf4d76b3b6015b96258ed0/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.260423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf7d54-3776-4d27-9969-d71393b9fe86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=432db4bcef9d4d769509e7344a6c1942, user_id=43cade108cdf4d76b3b6015b96258ed0, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.263783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf7d54-3776-4d27-9969-d71393b9fe86 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.274839 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b1549139e54022a4444bf2b22f27c4', 'request_id': 'req-e1ada915-0ea8-4623-8697-11a349550b61'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:00.291824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b1549139e54022a4444bf2b22f27c4', 'request_id': 'req-e1ada915-0ea8-4623-8697-11a349550b61'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:00.297975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 970/1946] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:00 2022] PUT /identity/v3/projects/432db4bcef9d4d769509e7344a6c1942/users/43cade108cdf4d76b3b6015b96258ed0/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:00.308824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b1549139e54022a4444bf2b22f27c4', 'request_id': 'req-e1ada915-0ea8-4623-8697-11a349550b61'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:00.319686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.320061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.320345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.340315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] The token cache is being invalidate because project e7b1549139e54022a4444bf2b22f27c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:00.341785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:00.343597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1ada915-0ea8-4623-8697-11a349550b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:00.346957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 977/1947] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:00 2022] DELETE /identity/v3/projects/e7b1549139e54022a4444bf2b22f27c4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:00.354291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff3970-8228-4d9c-9247-4a36fbecd024 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.356823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff3970-8228-4d9c-9247-4a36fbecd024 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.357417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff3970-8228-4d9c-9247-4a36fbecd024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.418800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] MFA Rules not processed for user `43cade108cdf4d76b3b6015b96258ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:00.420226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.444513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-03ff3970-8228-4d9c-9247-4a36fbecd024 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:00.447602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03ff3970-8228-4d9c-9247-4a36fbecd024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.503266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:00.540610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 971/1948] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:00.542773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.545592 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.607120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 978/1949] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:00.623091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.623483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.623838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.685643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:00.687643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:00.701348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 979/1950] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:00.706454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.722614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfece2-df63-4427-851c-44448353f7d5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.733218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.734758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.735064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.735382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.736239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:00.736561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.743829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.758481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42c665-c912-474d-a153-5ca32a2a084d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.779778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 980/1951] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:29:00.781014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.785165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.786015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.786290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.786765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.788691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.791656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.804161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288f47b-ce12-497b-a78a-383e44bd944a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:00.820377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:00.821070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:00.821369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:00.821655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:00.822380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:00.824368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:00.868282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 972/1952] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/projects => generated 387 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:00.873730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.877336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 981/1953] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:00.881628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:00.882002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:00.997506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31d32-1d4b-438d-8b29-f8056580711d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.028511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c13a7-411f-4d03-a3b9-2b1d2acf662a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.036616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.037933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.038526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.039256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.050492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.053654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.090555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 982/1954] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:01.092214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.110865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9979e02-5e75-4a7e-9002-2c40fe5e9c5c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.118572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.120039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.120487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.121137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.122803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.136609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.139798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.140904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.141751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.142211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.146030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.151360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.167033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 983/1955] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:01.169858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.191268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39128307-4dd6-42a9-a34d-c90adcc3c7b3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.213556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.214812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.215276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.215642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.217090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.221030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.239234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 973/1956] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:00 2022] POST /identity/v3/users => generated 300 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:01.241099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.244015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 984/1957] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:01.245504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.265603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c4995-0200-4fb2-a72f-6eec1ebefd5d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.271009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.272946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.273261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.273586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.274414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.276288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.290507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5ca5-79aa-43bd-89de-e8b358fd46ec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.300388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22f375-3abc-4e1c-931b-f4d6a5887c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.303636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22f375-3abc-4e1c-931b-f4d6a5887c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.304310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22f375-3abc-4e1c-931b-f4d6a5887c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.305067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22f375-3abc-4e1c-931b-f4d6a5887c5f admin admin] PATH_INFO: `/v3/projects/ef8f8f8535b64ee09fbcfb894522cb54/users/8226e73490f1451984003a5b68c91a46/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.359743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 974/1958] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:01.361894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.368451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22f375-3abc-4e1c-931b-f4d6a5887c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8f8f8535b64ee09fbcfb894522cb54, user_id=8226e73490f1451984003a5b68c91a46, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.374315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22f375-3abc-4e1c-931b-f4d6a5887c5f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.385277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c64ad-68f3-4359-8e98-a8ee02dcb0fd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.402224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.403148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.403438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.403738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] PATH_INFO: `/v3/projects/9cc760d0434c4e318b11846f455284cb/users/947768d3b6c64932b604b402e2a38025/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.413724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 985/1959] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:01 2022] PUT /identity/v3/projects/ef8f8f8535b64ee09fbcfb894522cb54/users/8226e73490f1451984003a5b68c91a46/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:01.429929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.430215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.430493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.439001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc760d0434c4e318b11846f455284cb, user_id=947768d3b6c64932b604b402e2a38025, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.469561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.493224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 975/1960] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:01 2022] PUT /identity/v3/projects/9cc760d0434c4e318b11846f455284cb/users/947768d3b6c64932b604b402e2a38025/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:01.496676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.534808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] MFA Rules not processed for user `8226e73490f1451984003a5b68c91a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:01.536544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:01.599643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:01.664105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e4603-af65-400a-b942-1c7736544cc7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.675818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.676815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.677206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.677586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.678974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.681361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.714283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 976/1961] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:01.719667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.728613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:01.739635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c1c57-6c26-4fd1-afb1-7fda62aca367 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.750653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 986/1962] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:01.752164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.755211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732102a2-ff8b-4c2c-8060-a2768247966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.757286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732102a2-ff8b-4c2c-8060-a2768247966e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.758135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732102a2-ff8b-4c2c-8060-a2768247966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.759140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732102a2-ff8b-4c2c-8060-a2768247966e admin admin] PATH_INFO: `/v3/projects/9cc760d0434c4e318b11846f455284cb/users/947768d3b6c64932b604b402e2a38025/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.767075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745c450-78ff-4871-89e2-41fad7356902 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.768254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732102a2-ff8b-4c2c-8060-a2768247966e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc760d0434c4e318b11846f455284cb, user_id=947768d3b6c64932b604b402e2a38025, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.773704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.774812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.774993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732102a2-ff8b-4c2c-8060-a2768247966e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.775393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.775586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.776690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.780256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.792602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 977/1963] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:01 2022] PUT /identity/v3/projects/9cc760d0434c4e318b11846f455284cb/users/947768d3b6c64932b604b402e2a38025/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:01.805403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55178385-4552-49af-b631-8c9522879c11 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.806265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55178385-4552-49af-b631-8c9522879c11 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.807230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55178385-4552-49af-b631-8c9522879c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.822450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 987/1964] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:01 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:01.827183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:01.856928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-55178385-4552-49af-b631-8c9522879c11 None None] MFA Rules not processed for user `947768d3b6c64932b604b402e2a38025`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:01.858438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-55178385-4552-49af-b631-8c9522879c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:01.945658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dda81-4aaa-4ba0-b6a4-4a4184bce9cd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:01.952362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:01.953472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:01.953762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:01.954044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:01.954762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:01.957727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:01.985215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 988/1965] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:01 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:01.989592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.000251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 978/1966] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:02.007167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71133c57-327a-40bf-81f4-b6c8814d3162 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:02.018781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55178385-4552-49af-b631-8c9522879c11 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.021636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-55178385-4552-49af-b631-8c9522879c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:02.024819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:02.026656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.029757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.032212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.032841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:02.038630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:02.064008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 989/1967] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:02.069196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.093179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e84f5-04ba-4011-b4ac-374339ee3cfc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:02.107587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:02.108506 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.109035 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.109328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] PATH_INFO: `/v3/projects/9d6d88e56dcf4d93907d8693fe6ff06e/users/3ad4749223314a90aea06cef4f2d483a/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.127034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55178385-4552-49af-b631-8c9522879c11 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:02.133326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:02.134368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.134832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.135285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.150562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:02.152577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:02.155399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6d88e56dcf4d93907d8693fe6ff06e, user_id=3ad4749223314a90aea06cef4f2d483a, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:02.158157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:02.166138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 979/1968] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:02.185142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 990/1969] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:02 2022] PUT /identity/v3/projects/9d6d88e56dcf4d93907d8693fe6ff06e/users/3ad4749223314a90aea06cef4f2d483a/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:02.189854 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.209927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed14d-2ab3-4a87-ab98-451ed789c580 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:02.262268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.352816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:02.369672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b9ada-e39a-48fe-9d04-d5c1eb2bb523 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:02.380860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:02.382022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.382327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.383296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] PATH_INFO: `/v3/users/43cade108cdf4d76b3b6015b96258ed0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.385870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43cade108cdf4d76b3b6015b96258ed0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:02.394125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:02.439410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:02.444677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.445632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.446356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.450494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:02.452326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:02.483030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 980/1970] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:02.489928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.538137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0967a2-f290-443f-a8b4-8df85e21006a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:02.551715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43cade108cdf4d76b3b6015b96258ed0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:02.558245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59150b-3caf-4fb0-859a-33c09ada098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:02.561073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59150b-3caf-4fb0-859a-33c09ada098e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.561447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59150b-3caf-4fb0-859a-33c09ada098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.561709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59150b-3caf-4fb0-859a-33c09ada098e admin admin] PATH_INFO: `/v3/projects/9d6d88e56dcf4d93907d8693fe6ff06e/users/3ad4749223314a90aea06cef4f2d483a/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.564553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43cade108cdf4d76b3b6015b96258ed0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:02.569048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59150b-3caf-4fb0-859a-33c09ada098e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6d88e56dcf4d93907d8693fe6ff06e, user_id=3ad4749223314a90aea06cef4f2d483a, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:02.590537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59150b-3caf-4fb0-859a-33c09ada098e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:02.602098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43cade108cdf4d76b3b6015b96258ed0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:02.603499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:02.604044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43cade108cdf4d76b3b6015b96258ed0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:02.622434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 981/1971] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:02 2022] PUT /identity/v3/projects/9d6d88e56dcf4d93907d8693fe6ff06e/users/3ad4749223314a90aea06cef4f2d483a/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:02.635239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.635534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.635816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.639076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-596505f5-9767-48be-b209-9004bc89d447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43cade108cdf4d76b3b6015b96258ed0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:02.669053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 991/1972] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:02 2022] DELETE /identity/v3/users/43cade108cdf4d76b3b6015b96258ed0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:02.671019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:02.673440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:02.673740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:02.703879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] MFA Rules not processed for user `3ad4749223314a90aea06cef4f2d483a`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:02.705711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:02.736151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:02.738935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:02.883948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 982/1973] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:02.885743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 992/1974] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:02.901506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.916283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:02.944918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:02.946459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:03.090061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6195a4e-135d-4860-ab6a-3d6cc4f5061b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:03.104325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:03.105906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:03.106199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:03.108085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:03.117364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5f43d-b835-4a9b-a356-5e1b0bd17bdb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:03.126520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:03.131262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:03.131803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:03.133527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:03.139305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:03.145245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:03.168926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:03.170944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:03.177475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 983/1975] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:03.200726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 993/1976] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:03.239048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:03.248947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e505fb-0ba2-44ac-90be-6d6853dea3ff service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:03.259459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:03.260141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:03.260427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:03.260705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:03.269033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:03.424641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:03.431266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:03.452338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 984/1977] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:04.311605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:04.314327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:04.346533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cef7cc-159a-44dc-bad6-354443ebb749 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:04.361319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:04.363644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:04.364217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:04.364705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] PATH_INFO: `/v3/projects/432db4bcef9d4d769509e7344a6c1942` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:04.367212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=432db4bcef9d4d769509e7344a6c1942)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:04.370747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:04.406051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:04.498387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432db4bcef9d4d769509e7344a6c1942', 'request_id': 'req-51955cd3-0bab-43cc-9c91-e282b7be9b9f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.507168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432db4bcef9d4d769509e7344a6c1942', 'request_id': 'req-51955cd3-0bab-43cc-9c91-e282b7be9b9f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.519245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432db4bcef9d4d769509e7344a6c1942', 'request_id': 'req-51955cd3-0bab-43cc-9c91-e282b7be9b9f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.543835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] The token cache is being invalidate because project 432db4bcef9d4d769509e7344a6c1942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:04.544136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.544855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.546811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 994/1978] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:04 2022] DELETE /identity/v3/projects/432db4bcef9d4d769509e7344a6c1942 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:04.550635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:04.564042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102aecf-30aa-4e33-bde8-3fff8d9752e9 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:04.595813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:04.660745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:04.661929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:04.662959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:04.663376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] PATH_INFO: `/v3/users/947768d3b6c64932b604b402e2a38025` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:04.665469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947768d3b6c64932b604b402e2a38025)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:04.669365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:04.757809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947768d3b6c64932b604b402e2a38025'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.773333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947768d3b6c64932b604b402e2a38025'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.789640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947768d3b6c64932b604b402e2a38025'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.790572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947768d3b6c64932b604b402e2a38025'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.805157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-4af36575-d5e4-4afc-a6fe-5a1243c8ed65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947768d3b6c64932b604b402e2a38025'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:04.815792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 985/1979] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:04 2022] DELETE /identity/v3/users/947768d3b6c64932b604b402e2a38025 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:05.213432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:05.219414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:05.252438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51955cd3-0bab-43cc-9c91-e282b7be9b9f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:05.395900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:05.397356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:05.397630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:05.398020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] PATH_INFO: `/v3/projects/9cc760d0434c4e318b11846f455284cb` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:05.407436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cc760d0434c4e318b11846f455284cb)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:05.411370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:05.499458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cc760d0434c4e318b11846f455284cb', 'request_id': 'req-22f17c7e-8663-4323-93cd-8739b12f566e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:05.509760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cc760d0434c4e318b11846f455284cb', 'request_id': 'req-22f17c7e-8663-4323-93cd-8739b12f566e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:05.520926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cc760d0434c4e318b11846f455284cb', 'request_id': 'req-22f17c7e-8663-4323-93cd-8739b12f566e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:05.536279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] The token cache is being invalidate because project 9cc760d0434c4e318b11846f455284cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:05.536279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:05.536862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-22f17c7e-8663-4323-93cd-8739b12f566e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:05.543002 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 995/1980] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:05 2022] DELETE /identity/v3/projects/9cc760d0434c4e318b11846f455284cb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:05.545053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:05.545348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:05.545630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:05.580974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:05.582429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:05.674659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 986/1981] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:05.675380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:05.675656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:05.675938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:05.721337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:05.721337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:05.726336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 996/1982] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:05.731063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:05.746746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466816f1-4b40-4b2d-b498-f262db61a05b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:05.752045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:05.752728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:05.753048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:05.753329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:05.754068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:05.754381 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:05.756674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:05.794442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 987/1983] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:29:05.799719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:05.811096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473ce74-20ef-43c7-84e0-b5ae6d29d726 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:05.816150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:05.816930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:05.817327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:05.817677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:05.818767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:05.820830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:05.848353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 997/1984] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:05 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:05.853411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:05.868358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187d8f6-4b55-4ef6-9db7-d62fc09d429b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:05.930927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:05.962337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:05.963062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:05.963418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:05.963726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:05.964426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:05.966607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:05.992336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 988/1985] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:05 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:05.994251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.000010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:06.006976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b0e41-c2e6-49c6-8ad1-0a5dc5e13728 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.021106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.022480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.022902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.023261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.024093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.025826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.039249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 998/1986] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:06.043431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.054768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679c270-b32f-4db7-9bac-074cb8ec8fe5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.060416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9260531-7c59-4d72-8a78-3eae002ab728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.061229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9260531-7c59-4d72-8a78-3eae002ab728 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.061584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9260531-7c59-4d72-8a78-3eae002ab728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.061937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9260531-7c59-4d72-8a78-3eae002ab728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.062749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9260531-7c59-4d72-8a78-3eae002ab728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.064448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9260531-7c59-4d72-8a78-3eae002ab728 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.076104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 989/1987] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:06.080299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.096092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d12ce-462e-4a24-af76-cc14871a8e4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.102312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.103158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.103520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.103722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] PATH_INFO: `/v3/projects/763e89cd3b9344b0b1f0fce8a12a7995/users/97961e635e944c098044c1417f44d944/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.127293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=763e89cd3b9344b0b1f0fce8a12a7995, user_id=97961e635e944c098044c1417f44d944, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.130060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.153770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 999/1988] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:06 2022] PUT /identity/v3/projects/763e89cd3b9344b0b1f0fce8a12a7995/users/97961e635e944c098044c1417f44d944/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:06.166281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.166752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.167107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.211031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] MFA Rules not processed for user `97961e635e944c098044c1417f44d944`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:06.212639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:06.295228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 990/1989] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:06.299854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.373455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200212ed-9e20-4e33-966c-be6f10be2b9f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.378360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.379032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.379330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.379622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.380310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.382761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.419023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1000/1990] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:06 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:06.423751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.437582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d18e8-4d58-4957-85bd-d6ac19e323f4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.523200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.523992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.524349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.524691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.525475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.527748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.552063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 991/1991] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:06 2022] POST /identity/v3/users => generated 307 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:06.553548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.564202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab86de-489e-4501-9137-8605f383b24b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.569958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.570624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.570914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.571193 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.571926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.573568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.584855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1001/1992] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:06.589059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.599210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fd0e9-b8c8-4273-aafc-30b093b5cef8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.604024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.604936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.605226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.605510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] PATH_INFO: `/v3/projects/d3aea777c7644340821d0e9c2e5ef06e/users/7b92c2de66bd4c268dc3ff35ba05679f/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.620720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3aea777c7644340821d0e9c2e5ef06e, user_id=7b92c2de66bd4c268dc3ff35ba05679f, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.623322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.636995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 992/1993] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:06 2022] PUT /identity/v3/projects/d3aea777c7644340821d0e9c2e5ef06e/users/7b92c2de66bd4c268dc3ff35ba05679f/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:06.641150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.652349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ecd61-d4b5-44ff-b337-d8bf9d3cfa5f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.711926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71198434-1bf5-4a3e-9d88-2ab5e9234cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.712631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198434-1bf5-4a3e-9d88-2ab5e9234cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.712927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198434-1bf5-4a3e-9d88-2ab5e9234cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.713212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198434-1bf5-4a3e-9d88-2ab5e9234cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.713947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71198434-1bf5-4a3e-9d88-2ab5e9234cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.715546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71198434-1bf5-4a3e-9d88-2ab5e9234cc5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.744301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1002/1994] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:06.744963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.756818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3be95-7101-4290-9d3f-4738c8925be7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:06.762127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:06.763031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.763314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.763631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] PATH_INFO: `/v3/projects/d3aea777c7644340821d0e9c2e5ef06e/users/7b92c2de66bd4c268dc3ff35ba05679f/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.765656 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3aea777c7644340821d0e9c2e5ef06e, user_id=7b92c2de66bd4c268dc3ff35ba05679f, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:06.768379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:06.773836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:06.783689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 993/1995] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:06 2022] PUT /identity/v3/projects/d3aea777c7644340821d0e9c2e5ef06e/users/7b92c2de66bd4c268dc3ff35ba05679f/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:06.794312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:06.794870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:06.795164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:06.824655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] MFA Rules not processed for user `7b92c2de66bd4c268dc3ff35ba05679f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:06.825908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:06.857945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:06.922546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1003/1996] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:06.934666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:06.936469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:07.016852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d796f-f322-4413-937b-bf854a5819ef admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:07.023815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:07.024502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:07.024822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:07.025290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:07.035700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:07.037192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:07.048341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 994/1997] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:07.153581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:07.172130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acc963-3851-4641-9a65-637e02fda899 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:07.176202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc4cc-e002-4abd-add3-31aa5279382a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:07.177058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc4cc-e002-4abd-add3-31aa5279382a service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:07.177778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc4cc-e002-4abd-add3-31aa5279382a service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:07.178478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc4cc-e002-4abd-add3-31aa5279382a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:07.294597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc4cc-e002-4abd-add3-31aa5279382a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:07.308443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc4cc-e002-4abd-add3-31aa5279382a service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:07.323343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1004/1998] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:07.664457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:07.668195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:07.811150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec084bd1-03fb-47a5-a72c-765738cedff2 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:07.816778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:07.821475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:07.821824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:07.822170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] PATH_INFO: `/v3/users/8226e73490f1451984003a5b68c91a46` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:07.823577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8226e73490f1451984003a5b68c91a46)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:07.825746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:07.884764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8226e73490f1451984003a5b68c91a46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:07.897853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8226e73490f1451984003a5b68c91a46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:07.919607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8226e73490f1451984003a5b68c91a46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:07.920375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8226e73490f1451984003a5b68c91a46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:07.939123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8226e73490f1451984003a5b68c91a46'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:07.953997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 995/1999] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:07 2022] DELETE /identity/v3/users/8226e73490f1451984003a5b68c91a46 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:07.959859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:07.961487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:07.961987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:08.007307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:08.008784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:08.119885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1005/2000] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:08.142435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:08.144235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:08.188222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:08.278569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc9eef-23fb-429a-888b-e8cdee269f2c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:08.291237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:08.291237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:08.291237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:08.291237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:08.296961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:08.298409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:08.310559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 996/2001] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:08.861367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:08.863470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:08.878455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2252ead-7576-4d11-8b9a-7818be5b5653 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:08.886560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:08.887443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:08.887728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:08.888004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] PATH_INFO: `/v3/projects/ef8f8f8535b64ee09fbcfb894522cb54` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:08.889594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef8f8f8535b64ee09fbcfb894522cb54)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:08.891528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:08.956140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef8f8f8535b64ee09fbcfb894522cb54', 'request_id': 'req-b2596a55-64c5-4d61-9850-1adfc9a7c834'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:08.961798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:08.963511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef8f8f8535b64ee09fbcfb894522cb54', 'request_id': 'req-b2596a55-64c5-4d61-9850-1adfc9a7c834'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:08.976225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef8f8f8535b64ee09fbcfb894522cb54', 'request_id': 'req-b2596a55-64c5-4d61-9850-1adfc9a7c834'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:08.993187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] The token cache is being invalidate because project ef8f8f8535b64ee09fbcfb894522cb54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:08.993500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:08.994313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:08.996542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1006/2002] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:08 2022] DELETE /identity/v3/projects/ef8f8f8535b64ee09fbcfb894522cb54 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:09.000306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:09.012812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35744813-0c8d-431b-ad97-aff386fc1269 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:09.108953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:09.110307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:09.110605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:09.111022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] PATH_INFO: `/v3/users/3ad4749223314a90aea06cef4f2d483a` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:09.112616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad4749223314a90aea06cef4f2d483a)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:09.115603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:09.185112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad4749223314a90aea06cef4f2d483a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.192586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad4749223314a90aea06cef4f2d483a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.207278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad4749223314a90aea06cef4f2d483a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.208423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad4749223314a90aea06cef4f2d483a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.219182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae500a87-3d9b-4968-9eca-41694b5dc6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad4749223314a90aea06cef4f2d483a'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.232471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 997/2003] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:08 2022] DELETE /identity/v3/users/3ad4749223314a90aea06cef4f2d483a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:09.610137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:09.611852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:09.635621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2596a55-64c5-4d61-9850-1adfc9a7c834 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:09.738983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:09.739900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:09.740227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:09.741123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] PATH_INFO: `/v3/projects/9d6d88e56dcf4d93907d8693fe6ff06e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:09.742539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6d88e56dcf4d93907d8693fe6ff06e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:09.744534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:09.803484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6d88e56dcf4d93907d8693fe6ff06e', 'request_id': 'req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.810307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6d88e56dcf4d93907d8693fe6ff06e', 'request_id': 'req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.819198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6d88e56dcf4d93907d8693fe6ff06e', 'request_id': 'req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.831761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] The token cache is being invalidate because project 9d6d88e56dcf4d93907d8693fe6ff06e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:09.832063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.832720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eec5388f-61fd-49b6-b8b7-7fcac4d926b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:09.835169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1007/2004] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:09 2022] DELETE /identity/v3/projects/9d6d88e56dcf4d93907d8693fe6ff06e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:09.840800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:09.841481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:09.842022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:09.863941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:09.865441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:09.917402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:09.961675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 998/2005] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:09.970897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:09.971385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:09.971495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:09.992113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:09.994005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:10.002530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1008/2006] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:10.007282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.023348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236f06d-66b0-4398-a368-1eb254ac7c92 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.028209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.029246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.029687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.030124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.030871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:10.031181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.033067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.048201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 999/2007] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:10.054760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.066074 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c3f44-8425-4bac-a6ab-06254c246e10 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.072116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.073044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.073450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.073645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.074349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.076454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.109716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1009/2008] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:10 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:10.114753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.127198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b4cfd-6f15-4fe8-840d-4b6925a6e921 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.209696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.211003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.211288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.211525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.212756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.215737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.248623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1000/2009] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:10 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:10.253377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.277068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653b6c-49fc-483f-a5c2-fe8ac2c494b5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.284039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.285345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.285808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.289440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.290612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.292461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.315264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1010/2010] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:10.319525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.331426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae110-2b16-462b-a24c-466fd5505717 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.338282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4476e8c-51ab-471e-87db-926bacc36a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.338282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4476e8c-51ab-471e-87db-926bacc36a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.338637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4476e8c-51ab-471e-87db-926bacc36a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.338786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4476e8c-51ab-471e-87db-926bacc36a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.339735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4476e8c-51ab-471e-87db-926bacc36a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.341457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4476e8c-51ab-471e-87db-926bacc36a3c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.363175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1001/2011] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:10.368614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.379580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c233d4e-e2fc-4822-9764-d71751ddd034 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.386763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.387693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.387980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.388261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] PATH_INFO: `/v3/projects/7964032108774eca980a479095d4aea4/users/e1355dd2b3d6497a987053e98f39c9ce/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.405757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7964032108774eca980a479095d4aea4, user_id=e1355dd2b3d6497a987053e98f39c9ce, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.408465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.428998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1011/2012] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:10 2022] PUT /identity/v3/projects/7964032108774eca980a479095d4aea4/users/e1355dd2b3d6497a987053e98f39c9ce/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:10.442344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.444076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.444677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.510914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] MFA Rules not processed for user `e1355dd2b3d6497a987053e98f39c9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:10.512283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:10.655198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1002/2013] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:10.660493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.694704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:10.774368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181a3b6-072c-4b63-99ba-848ff8cc59ef admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.793964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.793964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.793964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.793964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.794413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.794413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.828233 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1012/2014] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:10 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:10.829467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.849024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3195f-8f25-418f-a0e4-e311a7a456c8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.913334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:10.950528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.951765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.952246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.952724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.953945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.957340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.982219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1003/2015] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:10 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:10.986428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.998863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af15bb3-29cb-4fe0-a26b-d170950799be admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:11.004601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:11.004809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.005350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.006103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.009628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:11.009628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:11.021803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1013/2016] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:11.026622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:11.037513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e4ca5-e4aa-4835-ac65-4395893170f3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:11.042898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:11.043803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.044091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.044368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] PATH_INFO: `/v3/projects/1af101c7b90d4d07b0b2c03cd51e9064/users/50c49761b42b469188ccdc1077a511b9/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.062683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af101c7b90d4d07b0b2c03cd51e9064, user_id=50c49761b42b469188ccdc1077a511b9, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:11.065786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:11.086689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1004/2017] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:11 2022] PUT /identity/v3/projects/1af101c7b90d4d07b0b2c03cd51e9064/users/50c49761b42b469188ccdc1077a511b9/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:11.090741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:11.107356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0975727-3687-4e2d-b57f-6a9f15d8384a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:11.191606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2a008-f416-4b02-8d02-d531701a4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:11.192307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2a008-f416-4b02-8d02-d531701a4839 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.192597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2a008-f416-4b02-8d02-d531701a4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.192890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2a008-f416-4b02-8d02-d531701a4839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.194522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2a008-f416-4b02-8d02-d531701a4839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:11.197343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2a008-f416-4b02-8d02-d531701a4839 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:11.208008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1014/2018] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:11.213007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:11.225183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1696bd-2cb7-4ea1-b4c7-fe7c805099f7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:11.230375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:11.231280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.231565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.231840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] PATH_INFO: `/v3/projects/1af101c7b90d4d07b0b2c03cd51e9064/users/50c49761b42b469188ccdc1077a511b9/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.233932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af101c7b90d4d07b0b2c03cd51e9064, user_id=50c49761b42b469188ccdc1077a511b9, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:11.236633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:11.248025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1005/2019] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:11 2022] PUT /identity/v3/projects/1af101c7b90d4d07b0b2c03cd51e9064/users/50c49761b42b469188ccdc1077a511b9/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:11.260149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.260443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.260735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.292064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] MFA Rules not processed for user `50c49761b42b469188ccdc1077a511b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:11.292951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:11.370113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1015/2020] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:11.383322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:11.385184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:11.451038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3503778-4cce-47f9-b8c2-a969ee804aa5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:11.455113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:11.455779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.456090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.456385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.468303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:11.470005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:11.480766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1006/2021] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:11.604406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:11.606236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:11.624025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558c211-8d47-4bf2-a6bb-2cf1c75939e2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:11.629920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc396b-f901-4b4c-ab73-cb9bec63b389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:11.630608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc396b-f901-4b4c-ab73-cb9bec63b389 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:11.631490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc396b-f901-4b4c-ab73-cb9bec63b389 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:11.631871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc396b-f901-4b4c-ab73-cb9bec63b389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:11.637357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:11.712629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc396b-f901-4b4c-ab73-cb9bec63b389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:11.716366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc396b-f901-4b4c-ab73-cb9bec63b389 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:11.725974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1016/2022] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:12.160478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:12.163760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:12.246905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3806b-21bc-49db-93bd-3eea4a86604e service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:12.251508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:12.252396 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:12.252682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:12.253045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] PATH_INFO: `/v3/users/97961e635e944c098044c1417f44d944` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:12.254247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97961e635e944c098044c1417f44d944)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:12.256494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:12.294072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:12.310133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97961e635e944c098044c1417f44d944'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:12.317582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97961e635e944c098044c1417f44d944'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:12.330411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97961e635e944c098044c1417f44d944'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:12.331296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97961e635e944c098044c1417f44d944'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:12.340728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97961e635e944c098044c1417f44d944'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:12.347187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1007/2023] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:12 2022] DELETE /identity/v3/users/97961e635e944c098044c1417f44d944 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:12.352652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:12.353188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:12.353294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:12.371152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:12.372514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:12.439647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1017/2024] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:12.450315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:12.451787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:12.516861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a016e-16c6-435a-90ef-a3736aa6e66e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:12.520801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:12.522130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:12.522420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:12.522772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:12.533085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:12.535992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:12.546725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1008/2025] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:12.875927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:12.877573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:12.901436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea3cd4-292a-480d-b341-b7cc71595d5f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:12.907080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:12.908644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:12.909366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:12.910094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] PATH_INFO: `/v3/projects/763e89cd3b9344b0b1f0fce8a12a7995` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:12.911582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=763e89cd3b9344b0b1f0fce8a12a7995)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:12.914314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:12.981697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763e89cd3b9344b0b1f0fce8a12a7995', 'request_id': 'req-941631b1-8cbe-42e2-988d-abba0e90be5f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:12.988447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763e89cd3b9344b0b1f0fce8a12a7995', 'request_id': 'req-941631b1-8cbe-42e2-988d-abba0e90be5f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.001489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763e89cd3b9344b0b1f0fce8a12a7995', 'request_id': 'req-941631b1-8cbe-42e2-988d-abba0e90be5f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.016215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] The token cache is being invalidate because project 763e89cd3b9344b0b1f0fce8a12a7995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:13.016514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.017200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.019153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1018/2026] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:12 2022] DELETE /identity/v3/projects/763e89cd3b9344b0b1f0fce8a12a7995 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:13.023519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:13.048829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185832d3-d297-428c-965a-3983e5a593d9 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:13.127265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:13.128413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:13.128701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:13.129001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] PATH_INFO: `/v3/users/7b92c2de66bd4c268dc3ff35ba05679f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:13.130258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b92c2de66bd4c268dc3ff35ba05679f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:13.132203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:13.213326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b92c2de66bd4c268dc3ff35ba05679f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.220994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:13.233775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b92c2de66bd4c268dc3ff35ba05679f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.276368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b92c2de66bd4c268dc3ff35ba05679f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.277651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b92c2de66bd4c268dc3ff35ba05679f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.287687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8433dd84-8e45-4a20-87db-93163297df78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b92c2de66bd4c268dc3ff35ba05679f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.306584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1009/2027] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:13 2022] DELETE /identity/v3/users/7b92c2de66bd4c268dc3ff35ba05679f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:13.669045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:13.671634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:13.696455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941631b1-8cbe-42e2-988d-abba0e90be5f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:13.726786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:13.782672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:13.783657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:13.784322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:13.784613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] PATH_INFO: `/v3/projects/d3aea777c7644340821d0e9c2e5ef06e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:13.785863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3aea777c7644340821d0e9c2e5ef06e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:13.787987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:13.851379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3aea777c7644340821d0e9c2e5ef06e', 'request_id': 'req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.857641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3aea777c7644340821d0e9c2e5ef06e', 'request_id': 'req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.866615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3aea777c7644340821d0e9c2e5ef06e', 'request_id': 'req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.879804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] The token cache is being invalidate because project d3aea777c7644340821d0e9c2e5ef06e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:13.880285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.881605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc5e18ad-76fb-480b-ade2-c22d53d5d9f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:13.883517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1019/2028] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:13 2022] DELETE /identity/v3/projects/d3aea777c7644340821d0e9c2e5ef06e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:13.888285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:13.888584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:13.888862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:13.907037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:13.908263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:13.987138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1010/2029] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:13.995484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:13.995951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:13.996225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.016712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:14.018052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.026597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1020/2030] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:14.030689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.041182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc049f-a3c8-4153-bb22-f93fa041d813 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.045770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.046427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.046712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.046990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.047715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:14.048023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.049723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.062477 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1011/2031] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:14.066747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.076542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3200ab-e592-47d6-8023-6dcba7f2ff69 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.081936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.082629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.082910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.083185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.084462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.086596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.116418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1021/2032] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:14.121832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.132095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176702d-3a64-4681-8073-a85d25cf9411 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.188990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.189785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.190077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.190353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.191057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.193004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.215907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1012/2033] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:14.221195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.232160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0810c2a-8ca8-42ee-8fb3-2f58a26c078d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.236768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.237477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.237773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.238054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.238963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.241176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.253602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1022/2034] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:14.258654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.269056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010cb9-6b7b-4367-a100-5ef6b7b23757 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.273858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26059874-6b4e-418d-9119-04c8621328f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.274527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26059874-6b4e-418d-9119-04c8621328f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.274813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26059874-6b4e-418d-9119-04c8621328f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.275094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26059874-6b4e-418d-9119-04c8621328f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.275831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26059874-6b4e-418d-9119-04c8621328f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.277792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26059874-6b4e-418d-9119-04c8621328f0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.291428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1013/2035] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:14.294260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.305423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc82047-ba4a-44fa-a1dd-ff4b62815c01 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.310372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.311292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.311742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.312006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] PATH_INFO: `/v3/projects/6650251d7a814c5288240f6306a3b0c0/users/831d2d8675b244c1bd0a1fa6cb0f29e2/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.327440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6650251d7a814c5288240f6306a3b0c0, user_id=831d2d8675b244c1bd0a1fa6cb0f29e2, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.330160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.344110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1023/2036] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:14 2022] PUT /identity/v3/projects/6650251d7a814c5288240f6306a3b0c0/users/831d2d8675b244c1bd0a1fa6cb0f29e2/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:14.356218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.356518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.356805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.386179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] MFA Rules not processed for user `831d2d8675b244c1bd0a1fa6cb0f29e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:14.387402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.422905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:14.472002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1014/2037] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:14.477323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.523644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.525132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.547718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:14.551892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3db645-439e-4fc1-930b-ef429b8abb65 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.560575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.561315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.561972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.562251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.563139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.565150 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.594416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1024/2038] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:14.598462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.601498 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c0835-172d-4d22-9add-4a10695b3428 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.664097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.665120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.665520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.665888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] PATH_INFO: `/v3/users/e1355dd2b3d6497a987053e98f39c9ce` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.667172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1355dd2b3d6497a987053e98f39c9ce)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.667294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3dc35-c45c-4b4f-a6d1-bcd2881d0f66 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.669266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.672705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.673285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.673670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.673831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.674518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.676411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.711980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1025/2039] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:14.715946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.725141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1355dd2b3d6497a987053e98f39c9ce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:14.728271 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d30f6e-252c-4a6e-ac45-af0d30d84ec4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.732676 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1355dd2b3d6497a987053e98f39c9ce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:14.733370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.734059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.734371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.734663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.735359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.737350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.745941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1355dd2b3d6497a987053e98f39c9ce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:14.746723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1355dd2b3d6497a987053e98f39c9ce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:14.750309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1026/2040] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:14.750462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:14.752137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.756996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-39a19ef6-64ba-4d60-8b99-4c2f912673ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1355dd2b3d6497a987053e98f39c9ce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:14.763662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19e872-2842-476c-84c6-8be3a1342404 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.763957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1015/2041] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:14 2022] DELETE /identity/v3/users/e1355dd2b3d6497a987053e98f39c9ce => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:14.768187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.768481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.768761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.789413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:14.791045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.838333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.839346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.839630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.839909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.840656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.842337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.853943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1027/2042] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:14.864860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.868009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.882056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1016/2043] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:14.888180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c9f05-235f-41b4-8d93-6baccc65ce75 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.891968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:14.893449 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.893571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315fd7b-529c-4d9c-97ea-602bf7276627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:14.894374 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315fd7b-529c-4d9c-97ea-602bf7276627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.894713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315fd7b-529c-4d9c-97ea-602bf7276627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.894938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315fd7b-529c-4d9c-97ea-602bf7276627 admin admin] PATH_INFO: `/v3/projects/5f9349deeca544a9bfb9f9eb5202eaf4/users/e629fcc3bb0548fb8523e82b9dc88415/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.911796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315fd7b-529c-4d9c-97ea-602bf7276627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9349deeca544a9bfb9f9eb5202eaf4, user_id=e629fcc3bb0548fb8523e82b9dc88415, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:14.914940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315fd7b-529c-4d9c-97ea-602bf7276627 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:14.928090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1028/2044] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:14 2022] PUT /identity/v3/projects/5f9349deeca544a9bfb9f9eb5202eaf4/users/e629fcc3bb0548fb8523e82b9dc88415/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:14.938457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:14.938746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:14.939059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:14.955299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d54bb9-4894-4adb-b487-9e1fd90cb2a6 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:14.967069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] MFA Rules not processed for user `e629fcc3bb0548fb8523e82b9dc88415`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:14.968295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:14.993283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:15.012468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.013420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.013891 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.014323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.050534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1029/2045] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:15.055398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.086735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.088194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.092506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:15.100277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1017/2046] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:15.121536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b005137-bf81-4859-aa1f-ac4984d15a11 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.126509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.127176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.127493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.127905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.128616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.131346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.162872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1030/2047] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:15 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:15.164620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.175472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca627ab2-8a7d-4298-8359-ffd974d742af service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.243950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.244645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.244958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.245244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.245943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.247877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.276032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1018/2048] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:15 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:15.280550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.294093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31683967-26ee-46a6-99c1-894f4a445bed admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.298922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.299618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.299925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.300190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.300982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.302828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.316922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1031/2049] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:15.320032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.329874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04605-6ef4-4339-ba7e-b5713054c4ce admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.335384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.336286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.336575 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.336860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] PATH_INFO: `/v3/projects/154ea6565014422da3a98081631b7184/users/6215c9ad437e4fcf9a1ae2c842746c5e/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.352828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] RBAC: Authorizing `identity:create_grant(project_id=154ea6565014422da3a98081631b7184, user_id=6215c9ad437e4fcf9a1ae2c842746c5e, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.355895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.372137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1019/2050] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:15 2022] PUT /identity/v3/projects/154ea6565014422da3a98081631b7184/users/6215c9ad437e4fcf9a1ae2c842746c5e/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:15.376051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.387293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ae57b-0d0d-4142-93ec-d020952edaf6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.407631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.410248 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:15.467510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.468209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.468499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.468778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.469584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.471196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.483269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1032/2051] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:15.487387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.498401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572d785-46af-4b7a-ac90-42673e5595c3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.502268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b407-f0b4-4bc4-9bb7-9584cf03411f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.503910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acf21-154f-40fc-a044-18fa7cc29d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.505048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acf21-154f-40fc-a044-18fa7cc29d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.505345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acf21-154f-40fc-a044-18fa7cc29d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.506576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acf21-154f-40fc-a044-18fa7cc29d06 admin admin] PATH_INFO: `/v3/projects/154ea6565014422da3a98081631b7184/users/6215c9ad437e4fcf9a1ae2c842746c5e/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.508834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acf21-154f-40fc-a044-18fa7cc29d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154ea6565014422da3a98081631b7184, user_id=6215c9ad437e4fcf9a1ae2c842746c5e, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.511594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acf21-154f-40fc-a044-18fa7cc29d06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.515335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.516249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.516530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.516816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] PATH_INFO: `/v3/projects/7964032108774eca980a479095d4aea4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.518063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7964032108774eca980a479095d4aea4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.519986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.527148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1033/2052] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:15 2022] PUT /identity/v3/projects/154ea6565014422da3a98081631b7184/users/6215c9ad437e4fcf9a1ae2c842746c5e/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:15.540306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94706b01-721b-4671-b909-e1098f223f71 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.540713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94706b01-721b-4671-b909-e1098f223f71 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.540928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94706b01-721b-4671-b909-e1098f223f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.589848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-94706b01-721b-4671-b909-e1098f223f71 None None] MFA Rules not processed for user `6215c9ad437e4fcf9a1ae2c842746c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:15.591133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-94706b01-721b-4671-b909-e1098f223f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:15.596657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7964032108774eca980a479095d4aea4', 'request_id': 'req-5408c34b-83d3-43f4-8bf4-3c7a5938e136'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.597996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:15.606504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7964032108774eca980a479095d4aea4', 'request_id': 'req-5408c34b-83d3-43f4-8bf4-3c7a5938e136'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.618301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7964032108774eca980a479095d4aea4', 'request_id': 'req-5408c34b-83d3-43f4-8bf4-3c7a5938e136'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.620273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:15.636266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] The token cache is being invalidate because project 7964032108774eca980a479095d4aea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:15.636266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.638233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.641883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1020/2053] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:15 2022] DELETE /identity/v3/projects/7964032108774eca980a479095d4aea4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:15.645513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.647089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:15.684511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1034/2054] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:15.697297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94706b01-721b-4671-b909-e1098f223f71 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.700319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-94706b01-721b-4671-b909-e1098f223f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:15.734491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c34b-83d3-43f4-8bf4-3c7a5938e136 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.740550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.741640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.741969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.742255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] PATH_INFO: `/v3/users/50c49761b42b469188ccdc1077a511b9` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.743486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c49761b42b469188ccdc1077a511b9)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.745456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.768570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94706b01-721b-4671-b909-e1098f223f71 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.772962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.773615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.773939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.774261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.785677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:15.788070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:15.798495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1035/2055] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:15.801107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c49761b42b469188ccdc1077a511b9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.810164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c49761b42b469188ccdc1077a511b9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.826237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c49761b42b469188ccdc1077a511b9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.827243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c49761b42b469188ccdc1077a511b9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.835473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c49761b42b469188ccdc1077a511b9'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:15.841938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1021/2056] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:15 2022] DELETE /identity/v3/users/50c49761b42b469188ccdc1077a511b9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:15.852448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:15.854294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:15.870068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3057d-f29d-4997-becc-8dc5f1916f6f service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:15.940766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:15.941479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:15.941767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:15.942189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:15.944854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:16.016009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.017514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7446e-5f71-454d-b221-3a2855977aef service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.022536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:16.028056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1036/2057] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:16.173159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.174641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:16.254267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f862a2-fbff-4f03-ae3d-809dfa5309e9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.260778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.261710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.262000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.262278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] PATH_INFO: `/v3/projects/1af101c7b90d4d07b0b2c03cd51e9064` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.263493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af101c7b90d4d07b0b2c03cd51e9064)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.265470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.285078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:16.333684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af101c7b90d4d07b0b2c03cd51e9064', 'request_id': 'req-6d951521-8ced-482b-adc1-4bc01cb4ccce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:16.341590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af101c7b90d4d07b0b2c03cd51e9064', 'request_id': 'req-6d951521-8ced-482b-adc1-4bc01cb4ccce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:16.351902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af101c7b90d4d07b0b2c03cd51e9064', 'request_id': 'req-6d951521-8ced-482b-adc1-4bc01cb4ccce'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:16.364542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] The token cache is being invalidate because project 1af101c7b90d4d07b0b2c03cd51e9064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:16.365742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:16.366728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6d951521-8ced-482b-adc1-4bc01cb4ccce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:16.368788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1022/2058] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:16 2022] DELETE /identity/v3/projects/1af101c7b90d4d07b0b2c03cd51e9064 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:16.373678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.374009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.374297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.397507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:16.398805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:16.481111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1037/2059] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:16.489207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.489557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.490642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.511714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:16.513139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:16.521646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1023/2060] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:16.525916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.536147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8daedf-9696-45d8-90b5-ca870d102926 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.541673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.542326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.542607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.542882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.543752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:16.544060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.545738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.560386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1038/2061] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:16.564741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.574206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff9a2-5e87-4d72-a8e9-866010818fcb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.578823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.579485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.579768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.580062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.580758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.582703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.608310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1024/2062] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:16 2022] POST /identity/v3/projects => generated 407 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:16.612528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.623987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd2e2-0dc2-458b-8672-39337bbf0cb0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.683590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.684258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.684961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.684961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.685494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.691754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.713516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1039/2063] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:16 2022] POST /identity/v3/users => generated 304 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:16.714718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.726051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9f7ad-e8a1-4d1b-88cc-c684e35a69df admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.730915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.731583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.731872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.732153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.732909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.734525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.748313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1025/2064] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:16.752328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.763269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153984f-cbe8-46ed-8e5d-6f8e6986d30e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.767961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b0392-3b24-4526-8d39-deb927439f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.768625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0392-3b24-4526-8d39-deb927439f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.768935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0392-3b24-4526-8d39-deb927439f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.769234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b0392-3b24-4526-8d39-deb927439f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.769942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b0392-3b24-4526-8d39-deb927439f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.771816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b0392-3b24-4526-8d39-deb927439f14 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.782410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1040/2065] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:16.787224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:16.797911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28f4c9-2fe6-46dd-951c-e9a700df51e5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:16.803114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:16.804191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.804479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.804752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] PATH_INFO: `/v3/projects/81d27dc9ec0c44b1a84a6adc4409c3cc/users/3ae1544006ea4404acf2372390057462/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.821444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d27dc9ec0c44b1a84a6adc4409c3cc, user_id=3ae1544006ea4404acf2372390057462, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:16.824144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:16.838673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1026/2066] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:16 2022] PUT /identity/v3/projects/81d27dc9ec0c44b1a84a6adc4409c3cc/users/3ae1544006ea4404acf2372390057462/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:16.850070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dde25-a155-4792-808f-ab4481936bfd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:16.850416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dde25-a155-4792-808f-ab4481936bfd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:16.850764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dde25-a155-4792-808f-ab4481936bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:16.877650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-954dde25-a155-4792-808f-ab4481936bfd None None] MFA Rules not processed for user `3ae1544006ea4404acf2372390057462`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:16.878854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-954dde25-a155-4792-808f-ab4481936bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:16.940838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:16.952731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1041/2067] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:16.959130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.026657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0c3af-0759-4207-9dc2-5fc483ea50c5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.031452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.032117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.032406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.032690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.033442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.035613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.061643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1027/2068] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:16 2022] POST /identity/v3/projects => generated 409 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:17.065748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dde25-a155-4792-808f-ab4481936bfd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.077869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dde25-a155-4792-808f-ab4481936bfd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.137975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.138725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.139030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.139303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.139991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.142162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.164322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1042/2069] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:17 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:17.168507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.179768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd91c-234a-4d75-9524-6fca238ddb86 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.184508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.185231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.185516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.185792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.186516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.188225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.199980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1028/2070] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:17.204266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.216098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a711677-1ccf-40be-bff1-fac14d83266a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.220789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.221706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.221989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.222268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] PATH_INFO: `/v3/projects/cd35310155214ef0b066bc0090d70ce2/users/8c4eebd2bd9c40c98ff0120e3cf37660/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.239086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd35310155214ef0b066bc0090d70ce2, user_id=8c4eebd2bd9c40c98ff0120e3cf37660, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.242056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.256031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1043/2071] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:17 2022] PUT /identity/v3/projects/cd35310155214ef0b066bc0090d70ce2/users/8c4eebd2bd9c40c98ff0120e3cf37660/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:17.260203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.271464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4c7c2-e1a2-4328-9867-2077fdb92331 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.280869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:17.330924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd8eda-12e8-4168-becb-991e8468477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.331712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8eda-12e8-4168-becb-991e8468477f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.332070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8eda-12e8-4168-becb-991e8468477f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.332384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8eda-12e8-4168-becb-991e8468477f admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.333290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd8eda-12e8-4168-becb-991e8468477f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.335628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd8eda-12e8-4168-becb-991e8468477f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.346050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1029/2072] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:17.351538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.363219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89170265-018a-449f-a803-48ce0fa7f1bc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.368473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.369495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.369774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.370068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] PATH_INFO: `/v3/projects/cd35310155214ef0b066bc0090d70ce2/users/8c4eebd2bd9c40c98ff0120e3cf37660/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.372281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd35310155214ef0b066bc0090d70ce2, user_id=8c4eebd2bd9c40c98ff0120e3cf37660, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.375085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.387045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1044/2073] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:17 2022] PUT /identity/v3/projects/cd35310155214ef0b066bc0090d70ce2/users/8c4eebd2bd9c40c98ff0120e3cf37660/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:17.400109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.400403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.400675 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.428280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] MFA Rules not processed for user `8c4eebd2bd9c40c98ff0120e3cf37660`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:17.429468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:17.508902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1030/2074] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:17 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:17.520347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.523675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:17.607051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b608ce-d24a-4e6d-9f61-423259ea80dd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.610800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.611456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.611745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.612025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.632717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.634229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.646991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1045/2075] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:17 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:17.850990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:17.852631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:17.866116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e188-e350-4b66-96ea-f92e9987ed9c None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:17.870226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a80a2-5b1d-4bae-a1a6-09aef920862f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:17.873085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a80a2-5b1d-4bae-a1a6-09aef920862f service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:17.873085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a80a2-5b1d-4bae-a1a6-09aef920862f service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:17.873085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a80a2-5b1d-4bae-a1a6-09aef920862f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:17.936898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a80a2-5b1d-4bae-a1a6-09aef920862f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:17.938528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a80a2-5b1d-4bae-a1a6-09aef920862f service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:17.949749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1031/2076] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:20.078456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:20.079923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:20.143624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:20.160049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2ec27-e729-47c7-9434-a98c036bf52c service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:20.165349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:20.166261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:20.166544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:20.166821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] PATH_INFO: `/v3/users/831d2d8675b244c1bd0a1fa6cb0f29e2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:20.168020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831d2d8675b244c1bd0a1fa6cb0f29e2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:20.169976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:20.218411 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831d2d8675b244c1bd0a1fa6cb0f29e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.226085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831d2d8675b244c1bd0a1fa6cb0f29e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.239427 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831d2d8675b244c1bd0a1fa6cb0f29e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.240503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831d2d8675b244c1bd0a1fa6cb0f29e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.249219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831d2d8675b244c1bd0a1fa6cb0f29e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.261073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1046/2077] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:20 2022] DELETE /identity/v3/users/831d2d8675b244c1bd0a1fa6cb0f29e2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:20.267170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:20.267720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:20.268131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:20.290298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:20.291873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:20.330802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:20.363995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1032/2078] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:20.373884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:20.375443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:20.436047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c645-e5ed-43c6-b28c-24a4f7ecaf25 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:20.442161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:20.443345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:20.443764 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:20.444257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:20.453580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:20.455004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:20.465779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1047/2079] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:20.737022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:20.738942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:20.752389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19673877-ff92-4fd1-b099-3a974c317dcf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:20.758474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:20.759358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:20.759643 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:20.759920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] PATH_INFO: `/v3/projects/6650251d7a814c5288240f6306a3b0c0` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:20.761156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6650251d7a814c5288240f6306a3b0c0)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:20.763115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:20.811867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6650251d7a814c5288240f6306a3b0c0', 'request_id': 'req-f36e85f3-daed-4e02-9d21-23bfe2ff0736'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.817562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6650251d7a814c5288240f6306a3b0c0', 'request_id': 'req-f36e85f3-daed-4e02-9d21-23bfe2ff0736'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.825660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6650251d7a814c5288240f6306a3b0c0', 'request_id': 'req-f36e85f3-daed-4e02-9d21-23bfe2ff0736'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.838951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] The token cache is being invalidate because project 6650251d7a814c5288240f6306a3b0c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:20.839244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.839932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.841533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1033/2080] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:20 2022] DELETE /identity/v3/projects/6650251d7a814c5288240f6306a3b0c0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:20.845655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:20.855538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82660432-6a1e-4208-9b47-63b70b41797c service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:20.917228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:20.918107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:20.918417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:20.918719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] PATH_INFO: `/v3/users/e629fcc3bb0548fb8523e82b9dc88415` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:20.919988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e629fcc3bb0548fb8523e82b9dc88415)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:20.922101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:20.936855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:20.971654 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e629fcc3bb0548fb8523e82b9dc88415'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.979108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e629fcc3bb0548fb8523e82b9dc88415'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.991571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e629fcc3bb0548fb8523e82b9dc88415'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:20.992275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e629fcc3bb0548fb8523e82b9dc88415'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.001210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e629fcc3bb0548fb8523e82b9dc88415'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.009387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1048/2081] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:20 2022] DELETE /identity/v3/users/e629fcc3bb0548fb8523e82b9dc88415 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:21.307585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:21.309227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:21.322357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e85f3-daed-4e02-9d21-23bfe2ff0736 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:21.389405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:21.390291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:21.390576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:21.390851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] PATH_INFO: `/v3/projects/5f9349deeca544a9bfb9f9eb5202eaf4` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:21.392066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9349deeca544a9bfb9f9eb5202eaf4)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:21.393995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:21.452680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9349deeca544a9bfb9f9eb5202eaf4', 'request_id': 'req-26e04d9e-84c7-4fdf-8370-27b21c61cf36'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.460354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9349deeca544a9bfb9f9eb5202eaf4', 'request_id': 'req-26e04d9e-84c7-4fdf-8370-27b21c61cf36'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.462701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:21.469701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9349deeca544a9bfb9f9eb5202eaf4', 'request_id': 'req-26e04d9e-84c7-4fdf-8370-27b21c61cf36'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.484576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] The token cache is being invalidate because project 5f9349deeca544a9bfb9f9eb5202eaf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:21.484870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.485568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.487128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1034/2082] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:21 2022] DELETE /identity/v3/projects/5f9349deeca544a9bfb9f9eb5202eaf4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:21.491797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:21.493299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:21.507406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01845891-1f01-402c-ad6e-5f6f85c52475 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:21.580129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:21.581067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:21.581352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:21.581629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] PATH_INFO: `/v3/users/6215c9ad437e4fcf9a1ae2c842746c5e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:21.582969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6215c9ad437e4fcf9a1ae2c842746c5e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:21.586449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:21.639634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6215c9ad437e4fcf9a1ae2c842746c5e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.647629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6215c9ad437e4fcf9a1ae2c842746c5e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.664632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6215c9ad437e4fcf9a1ae2c842746c5e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.665687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6215c9ad437e4fcf9a1ae2c842746c5e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.674909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-950bf620-e301-40c6-ba16-1a370ef6e6bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6215c9ad437e4fcf9a1ae2c842746c5e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:21.681808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1049/2083] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:21 2022] DELETE /identity/v3/users/6215c9ad437e4fcf9a1ae2c842746c5e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:22.003475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.004982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.020285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e04d9e-84c7-4fdf-8370-27b21c61cf36 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.084825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.085606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.085908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.086181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] PATH_INFO: `/v3/projects/154ea6565014422da3a98081631b7184` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.087376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154ea6565014422da3a98081631b7184)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.089397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.139201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154ea6565014422da3a98081631b7184', 'request_id': 'req-24732abb-aacb-4902-b865-0eaedbbc4857'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.145032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154ea6565014422da3a98081631b7184', 'request_id': 'req-24732abb-aacb-4902-b865-0eaedbbc4857'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.153544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154ea6565014422da3a98081631b7184', 'request_id': 'req-24732abb-aacb-4902-b865-0eaedbbc4857'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.165388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] The token cache is being invalidate because project 154ea6565014422da3a98081631b7184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:22.165671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.166329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-24732abb-aacb-4902-b865-0eaedbbc4857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:22.167901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1035/2084] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:22 2022] DELETE /identity/v3/projects/154ea6565014422da3a98081631b7184 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:22.176188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.176478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.176784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.196120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:22.197283 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.203293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:22.274600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1050/2085] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:22.285118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.285245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.285494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.316399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:22.318033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.326457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1036/2086] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:22.330633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.341852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b639-7571-4403-8e0a-b25f571128c9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.346663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.347351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.347605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.347874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.348606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:22.348935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.351465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.364121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1051/2087] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:22.368948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.378324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8984-fc46-4d9e-a4ce-bbfedb7e9bf3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.383702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.384374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.384662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.384983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.387344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.389348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.420026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1037/2088] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:22.425097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.445542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a07f1-1e96-4a95-af53-fb39b5935b6d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.526026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.526732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.526943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.527220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.527927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.529914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.551879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1052/2089] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:22.556186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.568106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01e23-ea07-4c6e-bd80-0abe138eff48 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.574268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.579320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.579320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.579320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.579320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.579320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.591191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1038/2090] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:22.595690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.614324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d72c96-4a87-4de8-962c-75d83adfcf23 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.621076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243c588-78f6-4467-962a-7b010772fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.621717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243c588-78f6-4467-962a-7b010772fa4d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.622114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243c588-78f6-4467-962a-7b010772fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.622505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243c588-78f6-4467-962a-7b010772fa4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.624307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243c588-78f6-4467-962a-7b010772fa4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.626724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243c588-78f6-4467-962a-7b010772fa4d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.641339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1053/2091] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:22.645531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.658589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b923d-957d-4753-930b-9e7297a54906 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.665937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.666841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.667172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.667454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] PATH_INFO: `/v3/projects/a175385cd7b34f298e85de5e75066f26/users/61bea62eb05e46d4a35614031fe082ca/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.685993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a175385cd7b34f298e85de5e75066f26, user_id=61bea62eb05e46d4a35614031fe082ca, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.689708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.714079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1039/2092] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:22 2022] PUT /identity/v3/projects/a175385cd7b34f298e85de5e75066f26/users/61bea62eb05e46d4a35614031fe082ca/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:22.728245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.728563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.728838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.759212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] MFA Rules not processed for user `61bea62eb05e46d4a35614031fe082ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:22.760872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.831196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1054/2093] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:22.843112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.844826 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:22.859397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:22.906405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3ec9b-78aa-4be6-a84d-df39d59701bd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:22.910650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c0f6b-4575-4b97-9870-b34af2676cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:22.911535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c0f6b-4575-4b97-9870-b34af2676cee service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:22.912013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c0f6b-4575-4b97-9870-b34af2676cee service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:22.912348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c0f6b-4575-4b97-9870-b34af2676cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:22.922638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c0f6b-4575-4b97-9870-b34af2676cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:22.924246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c0f6b-4575-4b97-9870-b34af2676cee service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:22.935013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1040/2094] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:22.955280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:22.966251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:23.024239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f416b96-6552-412d-a87e-bf72a4def8b4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.028916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.029845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.030264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.030546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] PATH_INFO: `/v3/users/61bea62eb05e46d4a35614031fe082ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.031823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61bea62eb05e46d4a35614031fe082ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.033814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.081309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61bea62eb05e46d4a35614031fe082ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.089338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61bea62eb05e46d4a35614031fe082ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.102940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61bea62eb05e46d4a35614031fe082ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.103794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61bea62eb05e46d4a35614031fe082ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.113248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61bea62eb05e46d4a35614031fe082ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.120037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1055/2095] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:22 2022] DELETE /identity/v3/users/61bea62eb05e46d4a35614031fe082ca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:23.124798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.125142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.125430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.142833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:23.144042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.210470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1041/2096] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:23.220148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.221601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.281101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58e40f-a387-4e1b-b684-1a756fe23dc3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.286021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42497c01-3886-4c4c-9d02-c6ae48c3e911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.287853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42497c01-3886-4c4c-9d02-c6ae48c3e911 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.288146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42497c01-3886-4c4c-9d02-c6ae48c3e911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.288421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42497c01-3886-4c4c-9d02-c6ae48c3e911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.297885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42497c01-3886-4c4c-9d02-c6ae48c3e911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.299521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42497c01-3886-4c4c-9d02-c6ae48c3e911 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.312048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1056/2097] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:23.605880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.607673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.635858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bdbe6-ceb5-4b70-ad21-6c32c6b58334 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.642289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.643486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.643875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.644476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] PATH_INFO: `/v3/projects/a175385cd7b34f298e85de5e75066f26` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.645919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a175385cd7b34f298e85de5e75066f26)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.648375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.716478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a175385cd7b34f298e85de5e75066f26', 'request_id': 'req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.723149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a175385cd7b34f298e85de5e75066f26', 'request_id': 'req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.741755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a175385cd7b34f298e85de5e75066f26', 'request_id': 'req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.756092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] The token cache is being invalidate because project a175385cd7b34f298e85de5e75066f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:23.757278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.760079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e27a08bf-22d8-47d6-a9ac-fc3bd88ab412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:23.761983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1042/2098] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:23 2022] DELETE /identity/v3/projects/a175385cd7b34f298e85de5e75066f26 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:23.767079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.767797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.767797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.788135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:23.789509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.818597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:23.881264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1057/2099] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:23.884495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.884861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.885250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.906664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:23.908005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:23.919777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1043/2100] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:23.925241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.936249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252571-03c7-4464-a610-18a395cfa4c8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.942844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.944968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.944968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.945280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.946281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:23.946843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.949372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:23.970609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1058/2101] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:23.976900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:23.989368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f22295-891b-4f97-a376-42bfde73db24 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:23.994843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:23.995621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:23.995916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:23.996198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:23.996974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:23.999189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.026502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1044/2102] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:23 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:24.030481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.042591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02ca07-e668-4014-937d-30d737676f7a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.104076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.104683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.105009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.105292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.106666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.107926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.130085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1059/2103] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:24 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:24.134491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.146711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ffef6-2837-42d7-b5e5-4b2e56550d57 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.150950 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.151677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.152565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.152850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.154241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.155668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.162890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:24.173691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1045/2104] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:24.177155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.191180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859ca2e-10f1-478d-bf8a-d6544bbfeb38 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.197650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b051cf-c4bd-4e05-a19c-dfc5b246fee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.198328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b051cf-c4bd-4e05-a19c-dfc5b246fee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.198617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b051cf-c4bd-4e05-a19c-dfc5b246fee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.198893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b051cf-c4bd-4e05-a19c-dfc5b246fee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.199640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b051cf-c4bd-4e05-a19c-dfc5b246fee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.201294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b051cf-c4bd-4e05-a19c-dfc5b246fee2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.223535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1060/2105] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:24.225378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.235856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc619b19-cf71-4fd1-b26c-c7a9a8d85e4b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.241083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.241980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.242263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.242545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] PATH_INFO: `/v3/projects/ad180ca4a0d54be2b8077719c1cf21ab/users/d45c7457c32a493c95f7ebeddf8ed64b/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.260591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad180ca4a0d54be2b8077719c1cf21ab, user_id=d45c7457c32a493c95f7ebeddf8ed64b, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.263293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.277457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1046/2106] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:24 2022] PUT /identity/v3/projects/ad180ca4a0d54be2b8077719c1cf21ab/users/d45c7457c32a493c95f7ebeddf8ed64b/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:24.348436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849722d-af7d-4acc-993a-53ba509013da None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.348723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849722d-af7d-4acc-993a-53ba509013da None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.349138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849722d-af7d-4acc-993a-53ba509013da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.378018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c849722d-af7d-4acc-993a-53ba509013da None None] MFA Rules not processed for user `d45c7457c32a493c95f7ebeddf8ed64b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:24.379303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c849722d-af7d-4acc-993a-53ba509013da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:24.451179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1061/2107] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:24.464169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.466090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:24.530764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05edc5a-d4db-40d4-8b3f-83cdded12d1c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.535691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fb359-fb34-45a6-8c62-baaba5367f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.536305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fb359-fb34-45a6-8c62-baaba5367f52 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.536693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fb359-fb34-45a6-8c62-baaba5367f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.537056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fb359-fb34-45a6-8c62-baaba5367f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.549919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fb359-fb34-45a6-8c62-baaba5367f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.551377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fb359-fb34-45a6-8c62-baaba5367f52 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.561569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1047/2108] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:24.586525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849722d-af7d-4acc-993a-53ba509013da None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.615704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:24.658588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849722d-af7d-4acc-993a-53ba509013da None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.667970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.668908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.669197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.669476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] PATH_INFO: `/v3/users/d45c7457c32a493c95f7ebeddf8ed64b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.670729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d45c7457c32a493c95f7ebeddf8ed64b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.672727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:24.749408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd45c7457c32a493c95f7ebeddf8ed64b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:24.757457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd45c7457c32a493c95f7ebeddf8ed64b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:24.772487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd45c7457c32a493c95f7ebeddf8ed64b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:24.773384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd45c7457c32a493c95f7ebeddf8ed64b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:24.781509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd45c7457c32a493c95f7ebeddf8ed64b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:24.790773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1062/2109] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:24 2022] DELETE /identity/v3/users/d45c7457c32a493c95f7ebeddf8ed64b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:24.796672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5469b8-4619-4546-93a6-23121500135d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.797163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5469b8-4619-4546-93a6-23121500135d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.797329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5469b8-4619-4546-93a6-23121500135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.815603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cc5469b8-4619-4546-93a6-23121500135d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:24.816940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc5469b8-4619-4546-93a6-23121500135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:24.889583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1048/2110] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:24.900184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:24.901780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:24.974865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682bb0c-2b76-42b8-9a9e-5bf325cbaa2e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:24.981584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c97dda-a184-4fff-b279-b4cdde6986aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:24.982438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c97dda-a184-4fff-b279-b4cdde6986aa service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:24.982546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c97dda-a184-4fff-b279-b4cdde6986aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:24.982832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c97dda-a184-4fff-b279-b4cdde6986aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:24.994086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c97dda-a184-4fff-b279-b4cdde6986aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:24.995545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c97dda-a184-4fff-b279-b4cdde6986aa service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.010443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1063/2111] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:25.427182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5469b8-4619-4546-93a6-23121500135d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.428834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc5469b8-4619-4546-93a6-23121500135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:25.444588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5469b8-4619-4546-93a6-23121500135d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.450885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.451766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.452056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.452322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] PATH_INFO: `/v3/projects/ad180ca4a0d54be2b8077719c1cf21ab` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.453902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad180ca4a0d54be2b8077719c1cf21ab)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.456370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.513693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad180ca4a0d54be2b8077719c1cf21ab', 'request_id': 'req-3c421b2b-1c3e-409d-9484-96e134fb4664'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:25.519581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad180ca4a0d54be2b8077719c1cf21ab', 'request_id': 'req-3c421b2b-1c3e-409d-9484-96e134fb4664'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:25.527463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad180ca4a0d54be2b8077719c1cf21ab', 'request_id': 'req-3c421b2b-1c3e-409d-9484-96e134fb4664'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:25.540363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] The token cache is being invalidate because project ad180ca4a0d54be2b8077719c1cf21ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:25.540660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:25.541318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3c421b2b-1c3e-409d-9484-96e134fb4664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:25.542867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1049/2112] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:25 2022] DELETE /identity/v3/projects/ad180ca4a0d54be2b8077719c1cf21ab => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:25.547964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.548258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.548538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.569119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:25.570471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:25.638799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:25.663366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1064/2113] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:25.671444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.671662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.671937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.678711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:25.695266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:25.696503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:25.704688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1050/2114] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:25.709491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.719619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2657b2f-0009-4687-a78e-19c888cd595d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.724169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.724849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.725167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.725444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.726175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:25.726494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.728077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.739835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1065/2115] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:25.745689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.755685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f25d85-bae6-46ca-aa1d-f3d591c6f3f1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.767677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.768341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.768630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.768930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.769636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.772131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.801997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1051/2116] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:25 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:25.804147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.815133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe698d07-ee65-46b0-abcd-b9ede47acc91 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.875706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.876383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.876672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.876982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.877684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.879655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.901292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1066/2117] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:25 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:25.905393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.916144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc86dd-f661-4bc9-9840-ee287c9d6231 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.920858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.921594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.921833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.922113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.922850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.924451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.938403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1052/2118] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:25.943178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.953527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428d998-d322-4818-bf10-ac8183d09b06 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.958070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572494c2-705f-4c69-89fe-e436a9b7d685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.959422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572494c2-705f-4c69-89fe-e436a9b7d685 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.959615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572494c2-705f-4c69-89fe-e436a9b7d685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.959892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572494c2-705f-4c69-89fe-e436a9b7d685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:25.960617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572494c2-705f-4c69-89fe-e436a9b7d685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:25.962240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572494c2-705f-4c69-89fe-e436a9b7d685 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:25.972172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1067/2119] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:25.976641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:25.987200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cb326-d2dd-4271-a4bf-6b40b9a92961 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:25.992068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:25.993002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:25.993288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:25.993571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] PATH_INFO: `/v3/projects/2aa8c8dd804b46ca8996788a32b8383e/users/e73da7d7237d4c9cb11c60a84e4dccd8/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.009309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa8c8dd804b46ca8996788a32b8383e, user_id=e73da7d7237d4c9cb11c60a84e4dccd8, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:26.011960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:26.027761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1053/2120] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:25 2022] PUT /identity/v3/projects/2aa8c8dd804b46ca8996788a32b8383e/users/e73da7d7237d4c9cb11c60a84e4dccd8/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:26.032893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538cb8a-0ffb-4808-958c-9f674d3de592 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.033230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538cb8a-0ffb-4808-958c-9f674d3de592 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.033516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538cb8a-0ffb-4808-958c-9f674d3de592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.051419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e538cb8a-0ffb-4808-958c-9f674d3de592 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:26.053379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e538cb8a-0ffb-4808-958c-9f674d3de592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:26.115962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1068/2121] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:26.126602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:26.128635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:26.196558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cc8b-848c-4687-9a46-90fb1decc47b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:26.201121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:26.201778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.203718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.204011 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.215252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:26.216725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:26.229692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1054/2122] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:27.054755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.056103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.056701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.106824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] MFA Rules not processed for user `e73da7d7237d4c9cb11c60a84e4dccd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:27.107983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.164638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:27.196274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1069/2123] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:27.207263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.209455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.270261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffda13-c339-4420-81cc-e8b6f12510ec service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.274870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa316aa-20b5-4680-969a-3a6c28842cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.275526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa316aa-20b5-4680-969a-3a6c28842cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.275819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa316aa-20b5-4680-969a-3a6c28842cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.276099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa316aa-20b5-4680-969a-3a6c28842cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.285762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa316aa-20b5-4680-969a-3a6c28842cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.287965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa316aa-20b5-4680-969a-3a6c28842cb3 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.297577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1055/2124] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:27.605199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.606768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.633198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dffbf3-dc27-44a3-8305-eef74aff4acc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:27.633335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:27.634142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.634436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.634711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] PATH_INFO: `/v3/users/3ae1544006ea4404acf2372390057462` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.636117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae1544006ea4404acf2372390057462)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:27.638357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:27.732408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae1544006ea4404acf2372390057462'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:27.746890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae1544006ea4404acf2372390057462'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:27.784840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae1544006ea4404acf2372390057462'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:27.786133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae1544006ea4404acf2372390057462'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:27.799758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae1544006ea4404acf2372390057462'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:27.813604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1070/2125] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:27 2022] DELETE /identity/v3/users/3ae1544006ea4404acf2372390057462 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:27.815112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:27.815442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:27.815730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:27.836026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:27.837434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.896785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:27.919522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1056/2126] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:27.930412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:27.932118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:27.999021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad34d-e740-441e-b2e8-e8ab163d318e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:28.003043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:28.003695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:28.003982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:28.004263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:28.015922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:28.017642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:28.031118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1071/2127] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:28.344113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:28.345657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:28.360861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6f57b-ef8f-43de-a560-00942c36c9e2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:28.366519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:28.367839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:28.368237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:28.368868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] PATH_INFO: `/v3/projects/81d27dc9ec0c44b1a84a6adc4409c3cc` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:28.370645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d27dc9ec0c44b1a84a6adc4409c3cc)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:28.372810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:28.434874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d27dc9ec0c44b1a84a6adc4409c3cc', 'request_id': 'req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.442013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d27dc9ec0c44b1a84a6adc4409c3cc', 'request_id': 'req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.450856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d27dc9ec0c44b1a84a6adc4409c3cc', 'request_id': 'req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.465258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] The token cache is being invalidate because project 81d27dc9ec0c44b1a84a6adc4409c3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:28.465555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.466221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3526edfd-5a1d-48a7-ab31-a8bee2f21ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.468104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1057/2128] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:28 2022] DELETE /identity/v3/projects/81d27dc9ec0c44b1a84a6adc4409c3cc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:28.472971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:28.484417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bc8b3-bdcc-4ef5-89b3-8ab3ccf1b08d service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:28.545671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:28.553940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:28.555994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:28.556327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:28.556622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] PATH_INFO: `/v3/users/8c4eebd2bd9c40c98ff0120e3cf37660` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:28.559349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4eebd2bd9c40c98ff0120e3cf37660)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:28.560868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:28.616616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4eebd2bd9c40c98ff0120e3cf37660'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.626031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4eebd2bd9c40c98ff0120e3cf37660'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.638973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4eebd2bd9c40c98ff0120e3cf37660'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.639811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4eebd2bd9c40c98ff0120e3cf37660'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.655294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4eebd2bd9c40c98ff0120e3cf37660'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:28.668448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1072/2129] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:28 2022] DELETE /identity/v3/users/8c4eebd2bd9c40c98ff0120e3cf37660 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:28.790166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b028a-dcfd-42c7-829a-7f5de81b8719 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:28.790166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b028a-dcfd-42c7-829a-7f5de81b8719 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:28.790435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b028a-dcfd-42c7-829a-7f5de81b8719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:28.813578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f95b028a-dcfd-42c7-829a-7f5de81b8719 None None] MFA Rules not processed for user `e73da7d7237d4c9cb11c60a84e4dccd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:28.815052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f95b028a-dcfd-42c7-829a-7f5de81b8719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:28.941625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1058/2130] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:29.218998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.220585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:29.292116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50caa2d-7ed8-403d-9ba2-2dfb6d6ea8c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.296856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.297671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.297963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.298270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] PATH_INFO: `/v3/projects/cd35310155214ef0b066bc0090d70ce2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.299525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd35310155214ef0b066bc0090d70ce2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.301672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.356284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd35310155214ef0b066bc0090d70ce2', 'request_id': 'req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:29.362409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd35310155214ef0b066bc0090d70ce2', 'request_id': 'req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:29.370670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd35310155214ef0b066bc0090d70ce2', 'request_id': 'req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:29.382717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] The token cache is being invalidate because project cd35310155214ef0b066bc0090d70ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:29.383102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:29.383992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b500b5ec-6931-4fe0-bc13-ab7d3d1801d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:29.385732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1073/2131] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:29 2022] DELETE /identity/v3/projects/cd35310155214ef0b066bc0090d70ce2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:29.390567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.391069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.391164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.411398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:29.413101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:29.478088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1059/2132] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:29.486210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.486760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.486760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.501483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:29.509866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:29.511819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:29.523005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1074/2133] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:29.527283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.540576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b252e43-0007-49cd-8699-322f78c1c09a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.545648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.546316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.546605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.546883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.547620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:29.547928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.549581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.558649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:29.562602 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1060/2134] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:29.568142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.577661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c04da-8f16-4882-a1be-25f6cfb73910 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.582423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.583091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.583960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.584272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.585045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.586989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.614811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1075/2135] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:29.619600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.631277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b2e8f-ff02-4eac-9477-001c26ed79d3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.694170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.694747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.695032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.695310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.696006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.698027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.719872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1061/2136] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:29 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:29.724324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.735620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be89aa3-bac5-4503-9d46-8f3fa7f887f3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.741184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.741857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.742167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.742471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.743275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.745137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.757687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1076/2137] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:29.761831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.775351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adf8cf-9506-4b94-b2b4-09dfd396ef91 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.780718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f4201-968b-4c49-b21a-fd29f519aa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.781469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f4201-968b-4c49-b21a-fd29f519aa20 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.781815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f4201-968b-4c49-b21a-fd29f519aa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.782017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f4201-968b-4c49-b21a-fd29f519aa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.782784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f4201-968b-4c49-b21a-fd29f519aa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.784707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f4201-968b-4c49-b21a-fd29f519aa20 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.799216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1062/2138] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:29.804323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:29.815438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f50e10-905e-4ab7-92e1-6a5f045f837f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:29.820775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:29.821693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.821998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.822279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] PATH_INFO: `/v3/projects/9a601a7ea00f4972a2a4641c42a8670f/users/7d4121f0f4b34bfaba4e9bb86916fe8d/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.837323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a601a7ea00f4972a2a4641c42a8670f, user_id=7d4121f0f4b34bfaba4e9bb86916fe8d, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:29.839974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:29.853121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1077/2139] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:29 2022] PUT /identity/v3/projects/9a601a7ea00f4972a2a4641c42a8670f/users/7d4121f0f4b34bfaba4e9bb86916fe8d/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:29.863535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:29.863831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:29.864115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:29.893309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] MFA Rules not processed for user `7d4121f0f4b34bfaba4e9bb86916fe8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:29.894527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:29.967985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1063/2140] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:29.972744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.039208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f80c0-2c68-4d2c-89db-efc87f1e07c8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.044230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.044953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.045453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.045750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.046520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.048718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.079445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1078/2141] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:30.084048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.096517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c2264-46df-4b2a-95fd-1632d0a398ee None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.175737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.176539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.177523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.177523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.179245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.180620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.202975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:30.207905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1064/2142] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:30 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:30.219148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.231761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390a9bd-f86f-46ac-8f5f-19bad5351535 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.238863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.240100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.240100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.240100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.242021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.242995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.257199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1079/2143] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:30.261342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.273236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3acf8-8975-4ae8-b58e-ca4a126bb51f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.278954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.279861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.280144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.280419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] PATH_INFO: `/v3/projects/b71d0867b1a043249865c701f735c2c6/users/569b3e05fe1d4d68928a4a3138bad463/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.300718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71d0867b1a043249865c701f735c2c6, user_id=569b3e05fe1d4d68928a4a3138bad463, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.304005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.322816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1065/2144] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:30 2022] PUT /identity/v3/projects/b71d0867b1a043249865c701f735c2c6/users/569b3e05fe1d4d68928a4a3138bad463/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:30.327288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.339097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79156ab-c05c-42b7-9173-6a17d09ccc5b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.401492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bcd9b-2a21-4021-8c76-389ef8170652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.402372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bcd9b-2a21-4021-8c76-389ef8170652 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.405027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bcd9b-2a21-4021-8c76-389ef8170652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.405350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bcd9b-2a21-4021-8c76-389ef8170652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.406113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bcd9b-2a21-4021-8c76-389ef8170652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.407760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bcd9b-2a21-4021-8c76-389ef8170652 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.420815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1080/2145] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:30.424736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.437492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0043b-9cb9-40e2-b215-eb803527b30c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.444659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.445614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.445895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.446168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] PATH_INFO: `/v3/projects/b71d0867b1a043249865c701f735c2c6/users/569b3e05fe1d4d68928a4a3138bad463/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.448201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71d0867b1a043249865c701f735c2c6, user_id=569b3e05fe1d4d68928a4a3138bad463, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.450902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.465894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1066/2146] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:30 2022] PUT /identity/v3/projects/b71d0867b1a043249865c701f735c2c6/users/569b3e05fe1d4d68928a4a3138bad463/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:30.477357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.477604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.477914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.503672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:30.507751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] MFA Rules not processed for user `569b3e05fe1d4d68928a4a3138bad463`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:30.510421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:30.596312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1081/2147] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:30.607416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:30.609158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:30.668460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b3d21-147f-495c-b21f-b040edbaee3b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:30.672300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:30.673065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:30.673385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:30.673658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:30.736865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:30.738352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:30.749147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1067/2148] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:35.152264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:35.153843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:35.168492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71834d-2fc1-4913-a3c8-e67e147baaeb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:35.174085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:35.175629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:35.176130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:35.176372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:35.189493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:35.191587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:35.212587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1082/2149] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:36.550193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:36.551892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:36.632515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ad78-9c85-4fa1-a2fe-5b22377dbbf1 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:36.638254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:36.639394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:36.639747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:36.640123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] PATH_INFO: `/v3/users/a271ea90c6884817991aaa9e8be36cab` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:36.641778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a271ea90c6884817991aaa9e8be36cab)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:36.643886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:36.663544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:36.723032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a271ea90c6884817991aaa9e8be36cab'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.732581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a271ea90c6884817991aaa9e8be36cab'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.751293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a271ea90c6884817991aaa9e8be36cab'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.752096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a271ea90c6884817991aaa9e8be36cab'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.766200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0bd11533-fb21-4383-b605-5b9ddd6575dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a271ea90c6884817991aaa9e8be36cab'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:36.777393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1068/2150] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:36 2022] DELETE /identity/v3/users/a271ea90c6884817991aaa9e8be36cab => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:37.024437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.037405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e84f0-1654-4805-817e-195a1751c3ae service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.103249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.104146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.104430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.104712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] PATH_INFO: `/v3/projects/183d6a3ee79e4fedb83591ce60694283` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.105969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=183d6a3ee79e4fedb83591ce60694283)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.107900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.156562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183d6a3ee79e4fedb83591ce60694283', 'request_id': 'req-18ed6ec5-6a3e-4916-987f-0866ebb685f0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.163025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183d6a3ee79e4fedb83591ce60694283', 'request_id': 'req-18ed6ec5-6a3e-4916-987f-0866ebb685f0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.170353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183d6a3ee79e4fedb83591ce60694283', 'request_id': 'req-18ed6ec5-6a3e-4916-987f-0866ebb685f0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.182682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] The token cache is being invalidate because project 183d6a3ee79e4fedb83591ce60694283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:37.182992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.183636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18ed6ec5-6a3e-4916-987f-0866ebb685f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:37.185229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1083/2151] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:37 2022] DELETE /identity/v3/projects/183d6a3ee79e4fedb83591ce60694283 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:37.190470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.190762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.191041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.209443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:37.210706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.277664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1069/2152] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:37.287675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.288013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.288378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.307368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:37.308570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.316327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1084/2153] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:37.320625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.330642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc2d6-5c7b-475e-a297-5ddd8aefba72 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.336114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.336793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.337108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.337382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.338135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:37.338453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.340114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.354943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1070/2154] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:37.359361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.370273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da6333-5d6e-4dae-aff7-6d87dee6e656 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.376727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.377621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.377974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.378312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.379131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.381426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.402303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:37.436927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1085/2155] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:37.440007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.451028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9d585-ba97-4498-bb19-db96bf8186ab admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.527010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.527692 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.528123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.528398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.529126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.531164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.557360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1071/2156] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:37.558580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.569869 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b72d68-8080-4c1e-9725-2caf62a5823e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.574920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.575590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.575961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.576240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.577034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.578666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.590917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1086/2157] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:37.595278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.608518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae35e1-0143-4edc-8194-d7afdd21657f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.613346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8219b-aa5d-4720-9714-bc276718852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.614042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8219b-aa5d-4720-9714-bc276718852c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.614363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8219b-aa5d-4720-9714-bc276718852c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.614646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8219b-aa5d-4720-9714-bc276718852c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.615394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8219b-aa5d-4720-9714-bc276718852c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.617051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8219b-aa5d-4720-9714-bc276718852c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.622809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:37.634196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1072/2158] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:37.635159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.648410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd96d-eb70-46ca-af11-57cf81303193 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:37.653737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:37.654639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.654923 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.655205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] PATH_INFO: `/v3/projects/16b434ffed9f4a218fae3443c7cc7d31/users/a6f377b5eca84b60b2dac22f24840017/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.675450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b434ffed9f4a218fae3443c7cc7d31, user_id=a6f377b5eca84b60b2dac22f24840017, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:37.678610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:37.704173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1087/2159] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:37 2022] PUT /identity/v3/projects/16b434ffed9f4a218fae3443c7cc7d31/users/a6f377b5eca84b60b2dac22f24840017/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:37.716296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:37.716603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:37.716863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:37.748588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] MFA Rules not processed for user `a6f377b5eca84b60b2dac22f24840017`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:37.749917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:37.824193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1073/2160] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:37.932979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:37.934778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:38.023930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafce5d3-3c76-4dfe-b7aa-d4798f6375cf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:38.029905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53abf-42d0-4865-9f9b-942942840720 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:38.030569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53abf-42d0-4865-9f9b-942942840720 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:38.031091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53abf-42d0-4865-9f9b-942942840720 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:38.031476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53abf-42d0-4865-9f9b-942942840720 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:38.041803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53abf-42d0-4865-9f9b-942942840720 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:38.044003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53abf-42d0-4865-9f9b-942942840720 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:38.057620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1088/2161] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:44.592544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:44.594527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:44.672569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562fec9-65d8-4768-840f-2241944f2e1d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:44.679077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:44.679977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:44.680263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:44.680542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] PATH_INFO: `/v3/users/7d4121f0f4b34bfaba4e9bb86916fe8d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:44.681829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4121f0f4b34bfaba4e9bb86916fe8d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:44.683759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:44.743147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4121f0f4b34bfaba4e9bb86916fe8d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:44.751055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4121f0f4b34bfaba4e9bb86916fe8d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:44.763641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4121f0f4b34bfaba4e9bb86916fe8d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:44.764397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4121f0f4b34bfaba4e9bb86916fe8d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:44.773848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4121f0f4b34bfaba4e9bb86916fe8d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:44.782113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1074/2162] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:44 2022] DELETE /identity/v3/users/7d4121f0f4b34bfaba4e9bb86916fe8d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:44.787789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:44.788709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:44.789337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:44.812078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:44.813675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:44.888952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1089/2163] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:44.899413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:44.900899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:44.912909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:44.964684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784aa1-92fc-4a02-8b62-5f15f884364b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:44.968323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:44.969003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:44.969288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:44.969571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:44.979665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:44.981131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:44.991386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1075/2164] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:45.278236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:45.279722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:45.293535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be847-e2db-4954-b2a5-cfd8feb81cde None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:45.298349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:45.299214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:45.299498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:45.299776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] PATH_INFO: `/v3/projects/9a601a7ea00f4972a2a4641c42a8670f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:45.301046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a601a7ea00f4972a2a4641c42a8670f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:45.304152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:45.354668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:45.366574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a601a7ea00f4972a2a4641c42a8670f', 'request_id': 'req-eab51848-4e0b-478a-bb6a-4195c317b0d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.374383 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a601a7ea00f4972a2a4641c42a8670f', 'request_id': 'req-eab51848-4e0b-478a-bb6a-4195c317b0d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.388013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a601a7ea00f4972a2a4641c42a8670f', 'request_id': 'req-eab51848-4e0b-478a-bb6a-4195c317b0d0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.402365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] The token cache is being invalidate because project 9a601a7ea00f4972a2a4641c42a8670f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:45.402754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.403794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.405778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1090/2165] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:45 2022] DELETE /identity/v3/projects/9a601a7ea00f4972a2a4641c42a8670f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:45.409783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:45.423016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800230b-130c-40ab-bfe0-ecdd0c298fa3 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:45.506599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:45.507497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:45.507783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:45.508090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] PATH_INFO: `/v3/users/569b3e05fe1d4d68928a4a3138bad463` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:45.509376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=569b3e05fe1d4d68928a4a3138bad463)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:45.511354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:45.573391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '569b3e05fe1d4d68928a4a3138bad463'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.580565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '569b3e05fe1d4d68928a4a3138bad463'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.598712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '569b3e05fe1d4d68928a4a3138bad463'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.600260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '569b3e05fe1d4d68928a4a3138bad463'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.609553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c16a21c1-ec88-4903-a55d-3c9f1cbd7dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '569b3e05fe1d4d68928a4a3138bad463'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:45.618210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1076/2166] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:45 2022] DELETE /identity/v3/users/569b3e05fe1d4d68928a4a3138bad463 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:45.947214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:45.948920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:45.965141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab51848-4e0b-478a-bb6a-4195c317b0d0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.027219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.028083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.028368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.028645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] PATH_INFO: `/v3/projects/b71d0867b1a043249865c701f735c2c6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.029863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71d0867b1a043249865c701f735c2c6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.031779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.080244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71d0867b1a043249865c701f735c2c6', 'request_id': 'req-a39885fa-9875-469e-955c-71d61b47ea26'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:46.086293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71d0867b1a043249865c701f735c2c6', 'request_id': 'req-a39885fa-9875-469e-955c-71d61b47ea26'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:46.095120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71d0867b1a043249865c701f735c2c6', 'request_id': 'req-a39885fa-9875-469e-955c-71d61b47ea26'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:46.107488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] The token cache is being invalidate because project b71d0867b1a043249865c701f735c2c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:46.107685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:46.108318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a39885fa-9875-469e-955c-71d61b47ea26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:46.109885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1091/2167] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:45 2022] DELETE /identity/v3/projects/b71d0867b1a043249865c701f735c2c6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:46.116708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.117076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.117364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.137279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:46.139191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:46.168826 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:46.204569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1077/2168] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:46.213410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.213706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.213987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.233316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:46.234550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:46.243561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1092/2169] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:46.248020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.259759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c47be6-b8d2-4fea-ae51-87e16a7637ae None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.265274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.265945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.266229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.266535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.267275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:46.267584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.269368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.282443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1078/2170] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:46.287897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.297325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f01f0-f400-465c-9181-47068e111b68 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.302899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.303662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.304024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.304301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.305029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.306920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.340555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1093/2171] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:29:46.341929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.353832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbae2ff-4211-4e9f-8d2b-e3cb5b215acd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.410179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.410758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.411059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.411418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.412409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.414348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.435120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1079/2172] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:46.439568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.451814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b13d68-56a6-4353-baec-050609110865 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.457216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.457940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.458289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.458567 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.459324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.460983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.472911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1094/2173] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:46.477397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.487513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964d457-1a80-4345-a70f-613aababd51d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.493156 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a5e01-66e0-4252-aefc-01318b164ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.493821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5e01-66e0-4252-aefc-01318b164ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.494108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5e01-66e0-4252-aefc-01318b164ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.494385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5e01-66e0-4252-aefc-01318b164ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.495115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a5e01-66e0-4252-aefc-01318b164ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.496722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a5e01-66e0-4252-aefc-01318b164ec8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.508078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1080/2174] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:46.512368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.522873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426387a-fe9a-4a11-a251-437237f64328 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.527443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.528347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.528629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.528932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] PATH_INFO: `/v3/projects/5b2a1e39aa48438dbd27f683cce02a25/users/6e9e9d3e32cd4dd0876890cac6f47ba6/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.537542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:46.547293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2a1e39aa48438dbd27f683cce02a25, user_id=6e9e9d3e32cd4dd0876890cac6f47ba6, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.550064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.564376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1095/2175] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:46 2022] PUT /identity/v3/projects/5b2a1e39aa48438dbd27f683cce02a25/users/6e9e9d3e32cd4dd0876890cac6f47ba6/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:46.577739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.577739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.578025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.605947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] MFA Rules not processed for user `6e9e9d3e32cd4dd0876890cac6f47ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:46.607171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:46.702925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1081/2176] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:46.707815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.789908 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fad487-c627-46fd-a66e-fcd49d17b05c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.795840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.796658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.797037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.797388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.798159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.800158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.836665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1096/2177] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:46.840724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.857428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232cefc-ac73-4f8a-bfa2-7de4683f5b68 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:46.892760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:46.932501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:46.934651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:46.935058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:46.935809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:46.936608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:46.939867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:46.967050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1082/2178] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:46 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:46.970813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:46.999755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573dcf-c8da-416b-a5ea-3863c3074a3e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.006987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.009429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.009429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.009429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.009429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.011684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.023846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1097/2179] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:47.027823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.042521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368f68f-1a4d-4f1f-9b23-2cd751bfa337 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.047444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.048390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.048670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.049375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.049740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.051326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.064210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1083/2180] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:47.068760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.080688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c4881-4e26-440e-8b9a-7f49b2d41074 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.086555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e49aa-25ec-45fc-994c-8b0103532fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.087639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e49aa-25ec-45fc-994c-8b0103532fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.087849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e49aa-25ec-45fc-994c-8b0103532fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.088100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e49aa-25ec-45fc-994c-8b0103532fd0 admin admin] PATH_INFO: `/v3/projects/661843e8816a49ed928726e3f217065e/users/3c7b0f17d1e14458a8ceb7d02f681ea3/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.107549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e49aa-25ec-45fc-994c-8b0103532fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661843e8816a49ed928726e3f217065e, user_id=3c7b0f17d1e14458a8ceb7d02f681ea3, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.110621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e49aa-25ec-45fc-994c-8b0103532fd0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.142349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.144804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.147739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1098/2181] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:47 2022] PUT /identity/v3/projects/661843e8816a49ed928726e3f217065e/users/3c7b0f17d1e14458a8ceb7d02f681ea3/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:47.156482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.156779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.157140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.161895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9006a4-6564-4b54-aaa9-76907f73abb4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.189272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] MFA Rules not processed for user `3c7b0f17d1e14458a8ceb7d02f681ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:47.190435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.232206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.233962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.234251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.234574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] PATH_INFO: `/v3/users/a6f377b5eca84b60b2dac22f24840017` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.236456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f377b5eca84b60b2dac22f24840017)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.237972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.277671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1099/2182] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:47.278730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.291143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec4818-4a32-454a-8fad-b14f4473d4f4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.296786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.297487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.297776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.298056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.298752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.298880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f377b5eca84b60b2dac22f24840017'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:47.300657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.313429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f377b5eca84b60b2dac22f24840017'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:47.329258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1100/2183] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:47 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:47.333838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.342115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:47.342344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f377b5eca84b60b2dac22f24840017'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:47.343688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f377b5eca84b60b2dac22f24840017'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:47.353685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-61d735b0-087b-498e-a093-ed8ec9efaa3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f377b5eca84b60b2dac22f24840017'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:47.360166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1084/2184] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:47 2022] DELETE /identity/v3/users/a6f377b5eca84b60b2dac22f24840017 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:47.365368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.365942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.366483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.384625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:47.385894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.407136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:47.408510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026828f1-f33d-4076-829a-3460583f43b1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.457314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1085/2185] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:47.465352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.466927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.482454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.483245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.483613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.483904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.484689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.486697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.509498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1101/2186] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:47 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:47.513876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.515675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.528393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8931-88d3-41c1-8898-81514b891222 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.533252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.533919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.534202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.534740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b5e44-d0ed-4bbb-9340-b611f5cbca1b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.534858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.535465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.537784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.539784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.541023 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.541314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.541595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.551114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1102/2187] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:29:47.552699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.554586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.555452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.565056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1086/2188] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:47.566954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa04cb-6967-418d-8cbf-eec4b2a40316 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.573332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.574263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.574546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.574822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] PATH_INFO: `/v3/projects/080070d46d06489f9a3e4c29a8109c21/users/b4a3e9fce5e4429dbeb80d8a07a857e7/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.593197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=080070d46d06489f9a3e4c29a8109c21, user_id=b4a3e9fce5e4429dbeb80d8a07a857e7, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.595895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.610140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1103/2189] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:47 2022] PUT /identity/v3/projects/080070d46d06489f9a3e4c29a8109c21/users/b4a3e9fce5e4429dbeb80d8a07a857e7/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:47.617143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.628649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e90134-c3da-4478-9e86-0be966028e53 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.702279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19ee25-697b-4c9e-9797-6feb373cc3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.702985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ee25-697b-4c9e-9797-6feb373cc3af admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.703272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ee25-697b-4c9e-9797-6feb373cc3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.703552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ee25-697b-4c9e-9797-6feb373cc3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.704334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19ee25-697b-4c9e-9797-6feb373cc3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.706633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19ee25-697b-4c9e-9797-6feb373cc3af admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.721555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1087/2190] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:47.725935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.746277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5af922-e2e8-4ae1-91ac-80f636050ed5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.752966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.755677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.760085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.760381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] PATH_INFO: `/v3/projects/080070d46d06489f9a3e4c29a8109c21/users/b4a3e9fce5e4429dbeb80d8a07a857e7/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.768470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=080070d46d06489f9a3e4c29a8109c21, user_id=b4a3e9fce5e4429dbeb80d8a07a857e7, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.771779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:47.783003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1104/2191] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:47 2022] PUT /identity/v3/projects/080070d46d06489f9a3e4c29a8109c21/users/b4a3e9fce5e4429dbeb80d8a07a857e7/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:47.794418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.794729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.795117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.825104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] MFA Rules not processed for user `b4a3e9fce5e4429dbeb80d8a07a857e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:47.826315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.866127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.868615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.893274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:47.923910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1088/2192] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:47.935909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:47.937335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:47.945190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:47.960563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e98be9-786a-40f4-8e49-a2d6e826dd1d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:47.966297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:47.967183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:47.967482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:47.967765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] PATH_INFO: `/v3/projects/16b434ffed9f4a218fae3443c7cc7d31` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:47.968987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b434ffed9f4a218fae3443c7cc7d31)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:47.970968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.032489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b434ffed9f4a218fae3443c7cc7d31', 'request_id': 'req-43fa9feb-a279-40ff-ad15-caa1f933ca77'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:48.039575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b434ffed9f4a218fae3443c7cc7d31', 'request_id': 'req-43fa9feb-a279-40ff-ad15-caa1f933ca77'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:48.054335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257c39-3a18-420d-bc21-742a133cae09 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.054835 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b434ffed9f4a218fae3443c7cc7d31', 'request_id': 'req-43fa9feb-a279-40ff-ad15-caa1f933ca77'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:48.083967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] The token cache is being invalidate because project 16b434ffed9f4a218fae3443c7cc7d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:48.083967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:48.084400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-43fa9feb-a279-40ff-ad15-caa1f933ca77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:48.086140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1105/2193] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:47 2022] DELETE /identity/v3/projects/16b434ffed9f4a218fae3443c7cc7d31 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:48.093091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c90b-ba69-44d0-9b66-9a72a90390b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.093433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c90b-ba69-44d0-9b66-9a72a90390b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.093736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c90b-ba69-44d0-9b66-9a72a90390b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.132498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6669c90b-ba69-44d0-9b66-9a72a90390b1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:48.134019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6669c90b-ba69-44d0-9b66-9a72a90390b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:48.135645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.136562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.137043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.139388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.141204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:48.215050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1106/2194] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:48.221759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.222828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.223163 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.223485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.224432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.235598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1089/2195] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:48.246785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:48.248014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:48.257568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1107/2196] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:48.262141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.289933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cf3aa-2809-4d1c-9bea-2f0371027988 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.297826 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.298515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.298805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.299086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.299836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:48.300201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.302476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.315759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1090/2197] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:48.321490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.330895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df12ca-d472-4426-a170-61c01b7a780f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.336222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.336927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.337218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.337498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.338247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.340115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.372123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1108/2198] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:48.373741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.385502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7394dc6-872b-42d0-bfb8-2738a55718e2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.441031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:48.462032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.462910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.463304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.463688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.464491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.467913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.497096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1091/2199] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/users => generated 279 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:48.499186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.516290 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3127f26-a863-496c-be39-f0345c9d984b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.525211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.526301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.526778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.527347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.528662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.531804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.555413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1109/2200] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:48.556847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.571006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77544942-6481-4989-8358-f5931d2adf79 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.580944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdafe6b-360e-423e-8c30-6cb099ca67c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.582126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdafe6b-360e-423e-8c30-6cb099ca67c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.584925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdafe6b-360e-423e-8c30-6cb099ca67c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.585748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdafe6b-360e-423e-8c30-6cb099ca67c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.587033 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdafe6b-360e-423e-8c30-6cb099ca67c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.590772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdafe6b-360e-423e-8c30-6cb099ca67c5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.612982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1092/2201] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:48.616809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.627294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40e33-f236-4956-9cf1-fab75af1dfb7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.633326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.634402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.634754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.635103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] PATH_INFO: `/v3/projects/67627cf0bb804baf8f6cec1ab2c57d47/users/27e9797ee96f45b48ca4b1b64c0b4f29/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.652357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=67627cf0bb804baf8f6cec1ab2c57d47, user_id=27e9797ee96f45b48ca4b1b64c0b4f29, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.655128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.669929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1110/2202] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:48 2022] PUT /identity/v3/projects/67627cf0bb804baf8f6cec1ab2c57d47/users/27e9797ee96f45b48ca4b1b64c0b4f29/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:48.682619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.683105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.683221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.714342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] MFA Rules not processed for user `27e9797ee96f45b48ca4b1b64c0b4f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:48.715568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:48.786982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1093/2203] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:48.792864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.816543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:48.874163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06c170-229f-4a8a-a708-513c6d735dda admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:48.878962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:48.879733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:48.880306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:48.880306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:48.881006 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:48.882913 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:48.913694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1111/2204] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/projects => generated 357 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:48.919519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:48.932729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a38a87-da62-4ef7-830a-0d7792459270 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.012348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.013554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.014080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.014653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.015737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.019531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.051663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1094/2205] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:48 2022] POST /identity/v3/users => generated 285 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:49.053612 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.064958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b35d-0737-4d82-8aa4-72062f5eb4ca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.076170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.077029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.077416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.077692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.078490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.080485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.109031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1112/2206] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:49.111568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.125114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf5ee-db66-4f9b-a83c-e356509f7207 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.132008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a2e31-438b-422f-bf06-6f9adece9bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.133349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a2e31-438b-422f-bf06-6f9adece9bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.133349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a2e31-438b-422f-bf06-6f9adece9bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.133533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a2e31-438b-422f-bf06-6f9adece9bb3 admin admin] PATH_INFO: `/v3/projects/63c62e0d8cc54ed1ac8eca90441e4413/users/e0ba2aa708ef4ef1905ec9a227d5b169/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.152856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a2e31-438b-422f-bf06-6f9adece9bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c62e0d8cc54ed1ac8eca90441e4413, user_id=e0ba2aa708ef4ef1905ec9a227d5b169, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.155549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a2e31-438b-422f-bf06-6f9adece9bb3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.173875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1095/2207] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:49 2022] PUT /identity/v3/projects/63c62e0d8cc54ed1ac8eca90441e4413/users/e0ba2aa708ef4ef1905ec9a227d5b169/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:49.185073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.196328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970c995-5ecf-468b-9d59-e0f8ebfc4bcf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.249013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.249314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.249586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.266503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.269197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] MFA Rules not processed for user `493d126879f44b26951920235e7f7730`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:49.271168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:49.271582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.271914 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.272182 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.272924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.274711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.295204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1113/2208] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:49.295446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:49.296295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.313393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124c22b-740c-4912-8573-13ed96e3f185 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.320639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf33b10-72bf-4acd-9e31-953143ea8c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.322488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf33b10-72bf-4acd-9e31-953143ea8c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.322793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf33b10-72bf-4acd-9e31-953143ea8c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.323077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf33b10-72bf-4acd-9e31-953143ea8c24 admin admin] PATH_INFO: `/v3/projects/63c62e0d8cc54ed1ac8eca90441e4413/users/e0ba2aa708ef4ef1905ec9a227d5b169/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.325363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf33b10-72bf-4acd-9e31-953143ea8c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c62e0d8cc54ed1ac8eca90441e4413, user_id=e0ba2aa708ef4ef1905ec9a227d5b169, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.328499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf33b10-72bf-4acd-9e31-953143ea8c24 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.348360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1114/2209] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:49 2022] PUT /identity/v3/projects/63c62e0d8cc54ed1ac8eca90441e4413/users/e0ba2aa708ef4ef1905ec9a227d5b169/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:49.359509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.359920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.360474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.367844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1096/2210] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:49.391924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] MFA Rules not processed for user `e0ba2aa708ef4ef1905ec9a227d5b169`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:49.393816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:49.480656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1115/2211] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:49.493068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.496108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:49.563111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eec08c-2661-4469-be5e-9efe243b82bf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.618288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.619003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.619414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.619699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.631901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.634593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.648695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1097/2212] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:49.758059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.764658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:49.781450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4232a-8c6c-4be7-831c-112e63047bc9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.789208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.790175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.790554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.791031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.873225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:49.874952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:49.887144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1116/2213] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:49.899300 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:49.901665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:49.918304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b52f7-a759-4919-bdab-e9554f893b26 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:49.922646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:49.923327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:49.923616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:49.923952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:49.926065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:50.012235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:50.013878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:50.026534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1098/2214] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:50.766032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:50.777214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5b480-464f-4093-98a2-1a8eff18c10b service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:50.782141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:50.783011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:50.783372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:50.783788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:50.787085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:50.906159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:50.907880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c905e1-b9ec-4777-b510-720d53e3722f service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:50.923685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1117/2215] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:54.989714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:54.992382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:55.028406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:55.084162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc2ca6-cc90-4d63-87f5-b4690cbb0143 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:55.090690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:55.092275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:55.092275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:55.092483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] PATH_INFO: `/v3/users/6e9e9d3e32cd4dd0876890cac6f47ba6` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:55.094091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9e9d3e32cd4dd0876890cac6f47ba6)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:55.098176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:55.180996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9e9d3e32cd4dd0876890cac6f47ba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:55.188814 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9e9d3e32cd4dd0876890cac6f47ba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:55.206848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9e9d3e32cd4dd0876890cac6f47ba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:55.208566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9e9d3e32cd4dd0876890cac6f47ba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:55.230991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9e9d3e32cd4dd0876890cac6f47ba6'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:55.246392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1099/2216] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:54 2022] DELETE /identity/v3/users/6e9e9d3e32cd4dd0876890cac6f47ba6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:29:55.249032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:55.249501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:55.249973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:55.280469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:55.282049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:55.412582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1118/2217] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:29:55.420714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:55.422686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:55.513877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027988-a3d6-409e-86b4-41c2cbd85bc7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:55.522689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:55.523374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:55.523663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:55.523946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:55.580456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:55.582299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:55.618914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1100/2218] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:56.016025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:56.017599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:56.043857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5508-e28d-427b-815b-ef96b05504e5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:56.048627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:56.049836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:56.050123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:56.050399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] PATH_INFO: `/v3/projects/5b2a1e39aa48438dbd27f683cce02a25` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:56.053377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2a1e39aa48438dbd27f683cce02a25)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:56.055388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:56.105847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:56.122354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2a1e39aa48438dbd27f683cce02a25', 'request_id': 'req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.128232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2a1e39aa48438dbd27f683cce02a25', 'request_id': 'req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.136912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2a1e39aa48438dbd27f683cce02a25', 'request_id': 'req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.148961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] The token cache is being invalidate because project 5b2a1e39aa48438dbd27f683cce02a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:56.149195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.149864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.151429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1119/2219] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:56 2022] DELETE /identity/v3/projects/5b2a1e39aa48438dbd27f683cce02a25 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:56.155374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:56.165809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81b75c-781b-49e6-89a4-dc5891e531e0 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:56.207991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:56.231186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:56.232107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:56.232378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:56.232659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] PATH_INFO: `/v3/users/3c7b0f17d1e14458a8ceb7d02f681ea3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:56.233974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7b0f17d1e14458a8ceb7d02f681ea3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:56.236000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:56.292823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7b0f17d1e14458a8ceb7d02f681ea3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.300073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7b0f17d1e14458a8ceb7d02f681ea3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.312963 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7b0f17d1e14458a8ceb7d02f681ea3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.313691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7b0f17d1e14458a8ceb7d02f681ea3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.321650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7b0f17d1e14458a8ceb7d02f681ea3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.327781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1101/2220] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:56 2022] DELETE /identity/v3/users/3c7b0f17d1e14458a8ceb7d02f681ea3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:56.610546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:56.612114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:56.633138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d83a80-95dc-4f67-83ab-499b8a0aef8b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:56.713817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:56.714780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:56.715137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:56.715487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] PATH_INFO: `/v3/projects/661843e8816a49ed928726e3f217065e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:56.716900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] RBAC: Authorizing `identity:delete_project(project_id=661843e8816a49ed928726e3f217065e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:56.718826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:56.779763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661843e8816a49ed928726e3f217065e', 'request_id': 'req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.787648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661843e8816a49ed928726e3f217065e', 'request_id': 'req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.796718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661843e8816a49ed928726e3f217065e', 'request_id': 'req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.811647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] The token cache is being invalidate because project 661843e8816a49ed928726e3f217065e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:56.811890 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.812522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.814362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1120/2221] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:56 2022] DELETE /identity/v3/projects/661843e8816a49ed928726e3f217065e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:56.818204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:56.819836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:56.836731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c517fb-3025-427f-9136-e989a7bf32fd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:56.912634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:56.914198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:56.914865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:56.915355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] PATH_INFO: `/v3/users/b4a3e9fce5e4429dbeb80d8a07a857e7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:56.916751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a3e9fce5e4429dbeb80d8a07a857e7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:56.918932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:56.982384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a3e9fce5e4429dbeb80d8a07a857e7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:56.992972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a3e9fce5e4429dbeb80d8a07a857e7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.001702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:57.011590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a3e9fce5e4429dbeb80d8a07a857e7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.013027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a3e9fce5e4429dbeb80d8a07a857e7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.022411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5053610e-6ce8-4a48-9474-e718a52d01d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a3e9fce5e4429dbeb80d8a07a857e7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.029046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1102/2222] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:29:56 2022] DELETE /identity/v3/users/b4a3e9fce5e4429dbeb80d8a07a857e7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:57.372206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:57.373623 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:57.389025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fcdf1-5ade-4b0d-b082-b8a21d1f907f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:57.437541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:57.454849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:57.458434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.458434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.458434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] PATH_INFO: `/v3/projects/080070d46d06489f9a3e4c29a8109c21` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.458865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=080070d46d06489f9a3e4c29a8109c21)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:57.461346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:57.510747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '080070d46d06489f9a3e4c29a8109c21', 'request_id': 'req-3a52c57f-ad74-4616-b52d-386dde66a11d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.516581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '080070d46d06489f9a3e4c29a8109c21', 'request_id': 'req-3a52c57f-ad74-4616-b52d-386dde66a11d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.525113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '080070d46d06489f9a3e4c29a8109c21', 'request_id': 'req-3a52c57f-ad74-4616-b52d-386dde66a11d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.540581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] The token cache is being invalidate because project 080070d46d06489f9a3e4c29a8109c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:29:57.540797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.541484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3a52c57f-ad74-4616-b52d-386dde66a11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:29:57.543423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1121/2223] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:29:57 2022] DELETE /identity/v3/projects/080070d46d06489f9a3e4c29a8109c21 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:57.549981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.550486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.550591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.570821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:57.572059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:57.649498 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1103/2224] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:57.717360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.717935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.717935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.736718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:57.737976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:57.747745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1122/2225] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:57.752126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:57.762999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c8aa0-9a2c-4436-a49f-fe6d4e7e243e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:57.767969 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:57.768685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.769022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.769972 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.771250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:29:57.771629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:57.773308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:57.787236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1104/2226] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:29:57.791671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:57.801582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef7209-2899-4a7b-b375-f2225017d6db None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:57.807389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:57.808178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.808474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.808757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.809584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:57.811575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:57.841815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1123/2227] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:29:57 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:57.846225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:57.857331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd960c04-4938-42d3-9281-2d0f91f4047e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:57.923403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:57.924004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.924292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.924593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.925320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:57.927850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:57.952775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1105/2228] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:29:57 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:29:57.954252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:57.966718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c85ffd-a7a3-4e76-a319-88b2d0205983 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:57.977878 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:57.978583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:57.978863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:57.979153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:57.980033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:57.981840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:57.995247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1124/2229] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:58.001041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:58.013751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c1f32-4fa5-4c2d-b9a4-c01f6cfd6e43 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:58.018583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07b012-0687-43cf-a656-2d4a1711f42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:58.019993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07b012-0687-43cf-a656-2d4a1711f42b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:58.019993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07b012-0687-43cf-a656-2d4a1711f42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:58.019993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07b012-0687-43cf-a656-2d4a1711f42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:58.020775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07b012-0687-43cf-a656-2d4a1711f42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:58.022618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07b012-0687-43cf-a656-2d4a1711f42b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:58.036595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1106/2230] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:29:58.041065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:58.054932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe59bb-6a88-494c-867a-c00915da4386 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:58.060167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:58.061127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:58.061416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:58.061702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] PATH_INFO: `/v3/projects/91e1220a765f41e88d4e3480b9a7a687/users/43929f4dcce44f3bac9fd69cf960c985/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:58.104486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e1220a765f41e88d4e3480b9a7a687, user_id=43929f4dcce44f3bac9fd69cf960c985, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:58.107471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:58.124119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1125/2231] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:29:58 2022] PUT /identity/v3/projects/91e1220a765f41e88d4e3480b9a7a687/users/43929f4dcce44f3bac9fd69cf960c985/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:29:58.129733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eaf3e-b64a-4b36-9fcc-8d49e7b7a349 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:58.130046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eaf3e-b64a-4b36-9fcc-8d49e7b7a349 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:58.130308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eaf3e-b64a-4b36-9fcc-8d49e7b7a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:58.152355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-575eaf3e-b64a-4b36-9fcc-8d49e7b7a349 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:58.153628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-575eaf3e-b64a-4b36-9fcc-8d49e7b7a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:58.201484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:58.219565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1107/2232] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:58.234774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:58.236494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:58.324725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbd197-95e1-422d-a0f8-af0e4ad674a8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:58.333563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:58.334827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:58.335421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:58.336095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:58.350836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:58.352540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:58.363627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1126/2233] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:03.056651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.057370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.057370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.090873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] MFA Rules not processed for user `43929f4dcce44f3bac9fd69cf960c985`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:03.092227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:03.168279 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1108/2234] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:03.173098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.183685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cdd33-0afa-49df-b4e0-28d7dd3b958e service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.188279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.189061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.189418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.189759 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.190538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.193546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.207640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:03.226082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1127/2235] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:03 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:03.227151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.238373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67cc0f-c3e7-41d8-8286-dbcd46a1b34e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.317355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.317355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.317355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.317355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.318563 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.320639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.346471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1109/2236] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:03 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:03.350912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.362632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b285acb-1d2d-425b-acf6-509559fd903b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.371178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.371867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.372302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.372589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.373475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.375574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.388729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1128/2237] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:03.393018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.404718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69125131-96f2-4235-aa8a-5f94a4bbd92e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.410411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.412227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.412531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.412846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] PATH_INFO: `/v3/projects/2aff2e1ddb6f4c2e9818edc9babdd7f2/users/3e50a128b14743fa8d8cd38b4201ad6e/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.430328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aff2e1ddb6f4c2e9818edc9babdd7f2, user_id=3e50a128b14743fa8d8cd38b4201ad6e, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.433059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.478693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1110/2238] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:03 2022] PUT /identity/v3/projects/2aff2e1ddb6f4c2e9818edc9babdd7f2/users/3e50a128b14743fa8d8cd38b4201ad6e/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:03.482453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.499626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf459a2-b648-4ee9-a8fc-2aaaa49c553d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.564348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.565190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.565545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.565894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.566725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.569049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.583571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1129/2239] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:03.585399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:03.596407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842dc50-0b48-4ca9-af80-23edee4a958f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:03.602061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d796cf-2040-47f3-87d0-59a527092aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:03.602976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d796cf-2040-47f3-87d0-59a527092aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:03.603268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d796cf-2040-47f3-87d0-59a527092aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:03.603550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d796cf-2040-47f3-87d0-59a527092aa7 admin admin] PATH_INFO: `/v3/projects/2aff2e1ddb6f4c2e9818edc9babdd7f2/users/3e50a128b14743fa8d8cd38b4201ad6e/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:03.605693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d796cf-2040-47f3-87d0-59a527092aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aff2e1ddb6f4c2e9818edc9babdd7f2, user_id=3e50a128b14743fa8d8cd38b4201ad6e, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:03.612474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d796cf-2040-47f3-87d0-59a527092aa7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:03.631664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1111/2240] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:03 2022] PUT /identity/v3/projects/2aff2e1ddb6f4c2e9818edc9babdd7f2/users/3e50a128b14743fa8d8cd38b4201ad6e/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:06.316395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:06.319071 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:06.342666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69253e-0e81-4890-9996-9a3decd77447 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:06.524076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:06.525769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:06.526146 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:06.526924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] PATH_INFO: `/v3/users/27e9797ee96f45b48ca4b1b64c0b4f29` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:06.530473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e9797ee96f45b48ca4b1b64c0b4f29)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:06.543238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:06.650565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e9797ee96f45b48ca4b1b64c0b4f29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:06.668737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e9797ee96f45b48ca4b1b64c0b4f29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:06.690392 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e9797ee96f45b48ca4b1b64c0b4f29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:06.692147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e9797ee96f45b48ca4b1b64c0b4f29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:06.708149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e9797ee96f45b48ca4b1b64c0b4f29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:06.737286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:06.743287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1130/2241] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:06 2022] DELETE /identity/v3/users/27e9797ee96f45b48ca4b1b64c0b4f29 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:06.747860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:06.748357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:06.748453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:06.782586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:06.784405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:06.851559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:06.904462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1112/2242] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:06.912111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:06.913790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:07.008465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b6e63-0194-4f8d-8b70-ee086e8c89f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:07.012387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:07.013937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:07.014240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:07.014529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:07.027334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:07.029013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:07.045973 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1131/2243] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:07.581495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:07.583952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:07.616518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2137a7-c2ac-4ba6-a796-b06c2e7fdca5 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:07.626375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:07.627295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:07.627654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:07.628032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] PATH_INFO: `/v3/projects/67627cf0bb804baf8f6cec1ab2c57d47` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:07.634097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67627cf0bb804baf8f6cec1ab2c57d47)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:07.636605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:07.741400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67627cf0bb804baf8f6cec1ab2c57d47', 'request_id': 'req-30488cc8-ac21-4114-8f67-6b4ebaab2582'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:07.752211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67627cf0bb804baf8f6cec1ab2c57d47', 'request_id': 'req-30488cc8-ac21-4114-8f67-6b4ebaab2582'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:07.781119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67627cf0bb804baf8f6cec1ab2c57d47', 'request_id': 'req-30488cc8-ac21-4114-8f67-6b4ebaab2582'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:07.806793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] The token cache is being invalidate because project 67627cf0bb804baf8f6cec1ab2c57d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:07.807625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:07.809470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:07.811958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1113/2244] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:07 2022] DELETE /identity/v3/projects/67627cf0bb804baf8f6cec1ab2c57d47 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:07.815452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:07.837503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b1c6d-e3d8-4188-8c5c-b5deb658c059 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:07.944763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:07.946273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:07.946673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:07.947064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] PATH_INFO: `/v3/users/e0ba2aa708ef4ef1905ec9a227d5b169` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:07.948725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ba2aa708ef4ef1905ec9a227d5b169)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:07.950798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:08.022386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ba2aa708ef4ef1905ec9a227d5b169'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:08.030812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ba2aa708ef4ef1905ec9a227d5b169'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:08.048493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ba2aa708ef4ef1905ec9a227d5b169'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:08.049515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ba2aa708ef4ef1905ec9a227d5b169'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:08.085792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4f770d7-f65a-4155-a172-dd4b394e069d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ba2aa708ef4ef1905ec9a227d5b169'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:08.107172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1132/2245] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:07 2022] DELETE /identity/v3/users/e0ba2aa708ef4ef1905ec9a227d5b169 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:08.857239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:08.857873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:08.878159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30488cc8-ac21-4114-8f67-6b4ebaab2582 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:09.005421 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:09.006513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.006802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.007081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] PATH_INFO: `/v3/projects/63c62e0d8cc54ed1ac8eca90441e4413` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.008447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c62e0d8cc54ed1ac8eca90441e4413)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:09.011824 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:09.124561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c62e0d8cc54ed1ac8eca90441e4413', 'request_id': 'req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:09.136189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:09.138488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c62e0d8cc54ed1ac8eca90441e4413', 'request_id': 'req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:09.150799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c62e0d8cc54ed1ac8eca90441e4413', 'request_id': 'req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:09.196151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] The token cache is being invalidate because project 63c62e0d8cc54ed1ac8eca90441e4413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:09.196458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:09.197240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0f702bd6-1314-44ef-932f-7e1e4a5f7f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:09.199155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1114/2246] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:08 2022] DELETE /identity/v3/projects/63c62e0d8cc54ed1ac8eca90441e4413 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:09.203946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.205458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.208058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.256152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:09.259013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:09.305108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:09.392700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1133/2247] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:09.401847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.402142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.402452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.440358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:09.441686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:09.469809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1115/2248] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:09.471029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:09.491750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a51f18-25f7-4147-a5de-cca712c402bb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:09.509977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:09.510886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.511291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.511683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.512525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:09.512975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:09.516988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:09.595265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1134/2249] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:09.600272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:09.610533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e41d62-4d45-4170-80c2-4472e01122f0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:09.626390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:09.626390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.626390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.626914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.628231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:09.629685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:09.712130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1116/2250] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:09 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:09.712944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:09.730499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca5079-3e8c-4d70-a593-3e13a58477fc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:09.878821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:09.880104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.880508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.880818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.881629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:09.883664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:09.921456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1135/2251] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:09 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:09.925353 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:09.937098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f8d05-0bb8-4352-ad72-f2795bd73c76 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:09.943486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:09.944503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:09.944942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:09.945451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:09.946276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:09.948128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:09.971590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1117/2252] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:09.977359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.002256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb472a8-f08e-4816-896f-b8767568e892 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.012614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b7b5b-6b31-4272-bacb-5dc6fa2f5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.013691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b7b5b-6b31-4272-bacb-5dc6fa2f5840 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.014059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b7b5b-6b31-4272-bacb-5dc6fa2f5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.014595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b7b5b-6b31-4272-bacb-5dc6fa2f5840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.015946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b7b5b-6b31-4272-bacb-5dc6fa2f5840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.021312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b7b5b-6b31-4272-bacb-5dc6fa2f5840 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.044114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1136/2253] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:10.044220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.056222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c379d78-92fb-4d23-9176-b00cedf284a7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.062482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.064138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.064523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.064929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] PATH_INFO: `/v3/projects/ce947248e3d8416db9bb1af2c0fa8dbf/users/b8eb268c840148659a9cefda9d4bf5c3/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.091129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce947248e3d8416db9bb1af2c0fa8dbf, user_id=b8eb268c840148659a9cefda9d4bf5c3, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.094186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.124851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1118/2254] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:10 2022] PUT /identity/v3/projects/ce947248e3d8416db9bb1af2c0fa8dbf/users/b8eb268c840148659a9cefda9d4bf5c3/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:10.127334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.127628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.128627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.169879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] MFA Rules not processed for user `b8eb268c840148659a9cefda9d4bf5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:10.172118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:10.288794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1137/2255] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:10.294038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.402562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7468c31-d47f-48cb-b651-36fd16a7bee6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.409515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253f6c-353c-41e7-b110-c048991a90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.410625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253f6c-353c-41e7-b110-c048991a90f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.411736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253f6c-353c-41e7-b110-c048991a90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.412291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253f6c-353c-41e7-b110-c048991a90f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.413579 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3253f6c-353c-41e7-b110-c048991a90f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.416533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3253f6c-353c-41e7-b110-c048991a90f8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.475405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1119/2256] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:10 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:10.479400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.499605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea404f-bb6a-4436-a5a4-b5110c3251f3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.537786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:10.590736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.591032 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.591311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.630492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.630711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] MFA Rules not processed for user `3e50a128b14743fa8d8cd38b4201ad6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:10.631402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.631600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.631822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.631923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:10.632540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.634997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.672217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1138/2257] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:10 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:10.674315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.698417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c562-4b88-42ec-8459-46ef5701fc16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.704546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.704871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:10.705310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.705656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.706194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.707053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.710241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.728478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1139/2258] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:10.734005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.750075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1120/2259] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:10.752264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71ef24-2462-4f29-9871-e39997b26886 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.760671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.761614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.761931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.762164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] PATH_INFO: `/v3/projects/fdcecb8651364db1ab5f484a3f57a43c/users/8cc561d364864a098397f7e862113eb0/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.788840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcecb8651364db1ab5f484a3f57a43c, user_id=8cc561d364864a098397f7e862113eb0, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.791595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.807460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1140/2260] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:10 2022] PUT /identity/v3/projects/fdcecb8651364db1ab5f484a3f57a43c/users/8cc561d364864a098397f7e862113eb0/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:10.811479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.834845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e77ac-ca2e-4da8-9c8b-c93f1d0b416a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.910606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39ee2f-d46c-4724-b25b-16a7dd9d7520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.911691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ee2f-d46c-4724-b25b-16a7dd9d7520 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.912137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ee2f-d46c-4724-b25b-16a7dd9d7520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.912567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ee2f-d46c-4724-b25b-16a7dd9d7520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.913520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39ee2f-d46c-4724-b25b-16a7dd9d7520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.915390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39ee2f-d46c-4724-b25b-16a7dd9d7520 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.937626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1121/2261] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:10.942701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:10.953507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b64a8-1a7a-4c4a-b5de-934bebd6169c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:10.959740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:10.960851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.961255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.961716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] PATH_INFO: `/v3/projects/fdcecb8651364db1ab5f484a3f57a43c/users/8cc561d364864a098397f7e862113eb0/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:10.963774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcecb8651364db1ab5f484a3f57a43c, user_id=8cc561d364864a098397f7e862113eb0, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:10.967847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:10.983121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1141/2262] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:10 2022] PUT /identity/v3/projects/fdcecb8651364db1ab5f484a3f57a43c/users/8cc561d364864a098397f7e862113eb0/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:10.998965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:10.999411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:10.999798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:11.067630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] MFA Rules not processed for user `8cc561d364864a098397f7e862113eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:11.070098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:11.220624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1122/2263] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:11.232601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:11.234860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:11.310455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3f47-8786-4314-ba53-c331259e0c21 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:11.314398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8b3e8-1807-43ae-82f7-6ee8d9f8daa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:11.315052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8b3e8-1807-43ae-82f7-6ee8d9f8daa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:11.315344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8b3e8-1807-43ae-82f7-6ee8d9f8daa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:11.315625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8b3e8-1807-43ae-82f7-6ee8d9f8daa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:11.326091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8b3e8-1807-43ae-82f7-6ee8d9f8daa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:11.327610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8b3e8-1807-43ae-82f7-6ee8d9f8daa5 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:11.340500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1142/2264] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:11.526075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:11.622159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446210e-f348-4394-8d7b-87ce89d1c75b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:11.632586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:11.634185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:11.635245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:11.635801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] PATH_INFO: `/v3/users/b8eb268c840148659a9cefda9d4bf5c3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:11.638062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8eb268c840148659a9cefda9d4bf5c3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:11.641185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:11.662373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:11.724461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8eb268c840148659a9cefda9d4bf5c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:11.735104 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8eb268c840148659a9cefda9d4bf5c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:11.753117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8eb268c840148659a9cefda9d4bf5c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:11.753902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8eb268c840148659a9cefda9d4bf5c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:11.765262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8eb268c840148659a9cefda9d4bf5c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:11.802875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1123/2265] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:11 2022] DELETE /identity/v3/users/b8eb268c840148659a9cefda9d4bf5c3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:11.807202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951302c-619d-4a93-9522-a371d1375eca None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:11.807494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951302c-619d-4a93-9522-a371d1375eca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:11.807775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951302c-619d-4a93-9522-a371d1375eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:11.828797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1951302c-619d-4a93-9522-a371d1375eca None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:11.830458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1951302c-619d-4a93-9522-a371d1375eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:11.905402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1143/2266] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:11.916143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:11.918154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:11.991174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626c410-fbc9-4946-aa44-6a3356463462 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:11.996200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:11.997147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:11.997434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:11.997770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:12.022255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:12.028219 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:12.038192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1124/2267] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:12.522456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951302c-619d-4a93-9522-a371d1375eca None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:12.524104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1951302c-619d-4a93-9522-a371d1375eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:12.531863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:12.549130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951302c-619d-4a93-9522-a371d1375eca None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:12.583137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:12.584310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:12.584715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:12.585145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] PATH_INFO: `/v3/projects/ce947248e3d8416db9bb1af2c0fa8dbf` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:12.586490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce947248e3d8416db9bb1af2c0fa8dbf)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:12.588746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:12.596851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:12.605343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:12.627762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c732-d5f3-4d39-bbc9-5fc00dd2904d service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:12.632840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:12.633544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:12.633827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:12.634107 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:12.638547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:12.691027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce947248e3d8416db9bb1af2c0fa8dbf', 'request_id': 'req-26971e40-e4c2-4e87-8863-6eb218e375b2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:12.717222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce947248e3d8416db9bb1af2c0fa8dbf', 'request_id': 'req-26971e40-e4c2-4e87-8863-6eb218e375b2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:12.751264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce947248e3d8416db9bb1af2c0fa8dbf', 'request_id': 'req-26971e40-e4c2-4e87-8863-6eb218e375b2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:12.772169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] The token cache is being invalidate because project ce947248e3d8416db9bb1af2c0fa8dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:12.772590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:12.773395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:12.786131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1144/2268] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:12 2022] DELETE /identity/v3/projects/ce947248e3d8416db9bb1af2c0fa8dbf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:12.793892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:12.795823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:12.820389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:12.833589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:12.876237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1125/2269] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:12.976982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26971e40-e4c2-4e87-8863-6eb218e375b2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:12.992325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:12.993284 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:12.993645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:12.994249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] PATH_INFO: `/v3/users/8cc561d364864a098397f7e862113eb0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:12.995853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc561d364864a098397f7e862113eb0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:12.998120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:13.099672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc561d364864a098397f7e862113eb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.119228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc561d364864a098397f7e862113eb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.141051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc561d364864a098397f7e862113eb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.141950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc561d364864a098397f7e862113eb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.153027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-849961f1-fd07-4dc5-a500-bc5ab7f45790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc561d364864a098397f7e862113eb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.164716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1145/2270] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:12 2022] DELETE /identity/v3/users/8cc561d364864a098397f7e862113eb0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:13.522455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:13.536580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530009c6-1c97-4b96-849e-d75cec7c2023 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:13.582166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:13.634419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:13.635619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:13.635908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:13.636316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] PATH_INFO: `/v3/projects/fdcecb8651364db1ab5f484a3f57a43c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:13.637540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcecb8651364db1ab5f484a3f57a43c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:13.639514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:13.691588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcecb8651364db1ab5f484a3f57a43c', 'request_id': 'req-839640b9-2a55-4965-904f-35bd9cdf4b19'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.697725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcecb8651364db1ab5f484a3f57a43c', 'request_id': 'req-839640b9-2a55-4965-904f-35bd9cdf4b19'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.706052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcecb8651364db1ab5f484a3f57a43c', 'request_id': 'req-839640b9-2a55-4965-904f-35bd9cdf4b19'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.719711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] The token cache is being invalidate because project fdcecb8651364db1ab5f484a3f57a43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:13.720028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.720696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-839640b9-2a55-4965-904f-35bd9cdf4b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:13.723432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1126/2271] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:13 2022] DELETE /identity/v3/projects/fdcecb8651364db1ab5f484a3f57a43c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:13.728993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:13.729929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:13.730460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:13.739789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:13.756001 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:13.758273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:13.845322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1146/2272] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:13.853732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:13.854408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:13.854690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:13.874535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:13.875882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:13.886479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1127/2273] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:13.890452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:13.900738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105aa19-a5f2-4514-a8f0-61445e487af3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:13.905502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:13.906160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:13.906472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:13.906744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:13.907470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:13.907900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:13.909864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:13.922946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1147/2274] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:13.929167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:13.938725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4b4c-9ff8-4458-a5ca-66666069bb8e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:13.950097 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:13.950941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:13.951329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:13.951665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:13.952578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:13.954817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:13.982942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1128/2275] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:13 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:13.987565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:14.001747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79a5e4-af58-4e2d-a455-102fc133803a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:14.073504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:14.074179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:14.074465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:14.074737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:14.075436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:14.077452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:14.100958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1148/2276] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:13 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:14.102169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:14.114476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67126dac-4f1b-4a9b-8363-232bf800bab1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:14.119490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:14.120323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:14.120684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:14.120991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:14.121728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:14.123577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:14.135994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1129/2277] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:14.139756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:14.158653 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cc0c8-f266-4370-953d-fdbb29102971 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:14.164904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802b9a2-fda7-4d4d-a6de-5d5f785431ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:14.165642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802b9a2-fda7-4d4d-a6de-5d5f785431ce admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:14.165909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802b9a2-fda7-4d4d-a6de-5d5f785431ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:14.166185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802b9a2-fda7-4d4d-a6de-5d5f785431ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:14.167008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802b9a2-fda7-4d4d-a6de-5d5f785431ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:14.168820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802b9a2-fda7-4d4d-a6de-5d5f785431ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:14.183550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1149/2278] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:14.188245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:14.200566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9b8b-9a89-4a45-be43-e51ae40421e2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:14.206022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:14.206910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:14.207198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:14.207632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] PATH_INFO: `/v3/projects/88043393fe3f42bb9e89bb920ded5cb0/users/1a639e081ed64941aa3e440ad9ffbb7c/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:14.235570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=88043393fe3f42bb9e89bb920ded5cb0, user_id=1a639e081ed64941aa3e440ad9ffbb7c, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:14.239266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:14.259601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1130/2279] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:14 2022] PUT /identity/v3/projects/88043393fe3f42bb9e89bb920ded5cb0/users/1a639e081ed64941aa3e440ad9ffbb7c/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:14.264810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8620df-fe35-421b-9a73-8e971eb60436 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:14.265137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8620df-fe35-421b-9a73-8e971eb60436 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:14.265451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8620df-fe35-421b-9a73-8e971eb60436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:14.289713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ca8620df-fe35-421b-9a73-8e971eb60436 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:14.291303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca8620df-fe35-421b-9a73-8e971eb60436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:14.381796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1150/2280] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:14.390507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:14.392241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:14.464842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecdf16-52f4-49c7-b5c2-b08ec78ceebd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:14.469718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:14.470383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:14.470702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:14.470982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:14.482175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:14.483664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:14.488138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:14.496902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1131/2281] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:15.401103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:15.401582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:15.401849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:15.465883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:15.498010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] MFA Rules not processed for user `1a639e081ed64941aa3e440ad9ffbb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:15.500421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:15.589467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1151/2282] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:15.594109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:15.607751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85f06e-b9df-43d5-8e6d-b4b942670b3f service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:15.613218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:15.614762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:15.615379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:15.615896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:15.616833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:15.619255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:15.657664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1132/2283] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:15 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:15.658415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:15.682736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9917ee-ff0b-40b8-9749-1c8a0c372aef None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:15.762979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:15.763852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:15.765316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:15.765316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:15.766169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:15.786819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:15.811752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1152/2284] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:15 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:15.815868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:15.829749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d350f5d-7a83-4ab5-99a7-c3172e6042c2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:15.836136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:15.836833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:15.837147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:15.837815 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:15.838994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:15.840758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:15.855231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1133/2285] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:15.859292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:15.871474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b9ce-5564-4d81-9285-517487648375 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:15.880998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:15.883332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:15.885231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:15.886302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] PATH_INFO: `/v3/projects/cfd23bc3225942f9b8e8b021d46c572c/users/8f3ed63aee294774a88636c9e392e29d/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:15.914952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd23bc3225942f9b8e8b021d46c572c, user_id=8f3ed63aee294774a88636c9e392e29d, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:15.917805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:15.938226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1153/2286] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:15 2022] PUT /identity/v3/projects/cfd23bc3225942f9b8e8b021d46c572c/users/8f3ed63aee294774a88636c9e392e29d/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:15.943895 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:15.957624 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd08b-6244-42e9-926a-b6f869e2993a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:16.056237 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:16.057894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:16.058185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:16.058471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:16.059358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:16.061644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:16.084628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1134/2287] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:16.089424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:16.110484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3658aa1-a56d-4344-b0b3-04939befc281 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:16.116536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ec6ed-1e1f-490f-ac97-f9d43af61a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:16.117494 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ec6ed-1e1f-490f-ac97-f9d43af61a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:16.117949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ec6ed-1e1f-490f-ac97-f9d43af61a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:16.118206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ec6ed-1e1f-490f-ac97-f9d43af61a25 admin admin] PATH_INFO: `/v3/projects/cfd23bc3225942f9b8e8b021d46c572c/users/8f3ed63aee294774a88636c9e392e29d/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:16.120230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ec6ed-1e1f-490f-ac97-f9d43af61a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd23bc3225942f9b8e8b021d46c572c, user_id=8f3ed63aee294774a88636c9e392e29d, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:16.122893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ec6ed-1e1f-490f-ac97-f9d43af61a25 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:16.136938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1154/2288] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:16 2022] PUT /identity/v3/projects/cfd23bc3225942f9b8e8b021d46c572c/users/8f3ed63aee294774a88636c9e392e29d/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:16.802446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:16.804226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:16.909340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc3d90-1167-4e64-ac77-20842eb1b608 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.002554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.003334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.004406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.004724 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.009337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.093324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:17.147630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.150246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f709b-1bda-4c28-bf5a-f9670abf59bb service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:17.170924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1135/2289] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:17.246785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.247297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.247651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.281986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] MFA Rules not processed for user `8f3ed63aee294774a88636c9e392e29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:17.283372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.338934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:17.351106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.351678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.352021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.375203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1155/2290] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:17 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:17.375711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:17.377066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.387661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:17.389114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.451525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1136/2291] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:30:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:17.469919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ad4a-c534-4120-bbad-18f9498e4c97 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.474830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.475745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.476189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.476617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.518622 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:17.521901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:17.557058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.559249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:17.574267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1156/2292] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:17.616842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d5f00-287e-412d-9537-d2c9569fef18 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:17.623050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ba1f-c285-4f2f-aea7-0f616f92ef98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:17.623718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ba1f-c285-4f2f-aea7-0f616f92ef98 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:17.624010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ba1f-c285-4f2f-aea7-0f616f92ef98 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:17.624292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ba1f-c285-4f2f-aea7-0f616f92ef98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:17.634172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04ba1f-c285-4f2f-aea7-0f616f92ef98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:17.635709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04ba1f-c285-4f2f-aea7-0f616f92ef98 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:17.655208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1137/2293] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:30:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:30:19.257777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:19.259875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:19.273326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e886-e236-4d93-9738-ae4bafc1a76b service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:19.279305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817703a1-66cd-4ccc-84ce-3211423b35dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:19.280057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817703a1-66cd-4ccc-84ce-3211423b35dc service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:19.280636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817703a1-66cd-4ccc-84ce-3211423b35dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:19.280991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817703a1-66cd-4ccc-84ce-3211423b35dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:19.293340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817703a1-66cd-4ccc-84ce-3211423b35dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:19.294910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817703a1-66cd-4ccc-84ce-3211423b35dc service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:19.310598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1157/2294] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:27.626134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:27.626726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:27.626726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:27.646683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] MFA Rules not processed for user `43929f4dcce44f3bac9fd69cf960c985`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:27.648359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:27.661357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1138/2295] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:29.305524 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=64 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:30.442050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f12bd-57c3-45e6-8072-81fe56c5346b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.442685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f12bd-57c3-45e6-8072-81fe56c5346b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.442685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f12bd-57c3-45e6-8072-81fe56c5346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.464092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5d5f12bd-57c3-45e6-8072-81fe56c5346b None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:30.465668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5d5f12bd-57c3-45e6-8072-81fe56c5346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:30.474591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1158/2296] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:30:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:30.492238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:30.506417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e83aba-3706-4952-bd68-4a397063e923 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:30.513952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a3892-7474-4457-baaf-767d80d62257 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:30.514835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a3892-7474-4457-baaf-767d80d62257 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:30.515326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a3892-7474-4457-baaf-767d80d62257 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:30.515748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a3892-7474-4457-baaf-767d80d62257 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:30.527852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a3892-7474-4457-baaf-767d80d62257 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:30.529613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a3892-7474-4457-baaf-767d80d62257 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:30.543821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1139/2297] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:30:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:30:33.849112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7707e-2dc0-486b-af89-1c6ba876f0e7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.850097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7707e-2dc0-486b-af89-1c6ba876f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.850611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7707e-2dc0-486b-af89-1c6ba876f0e7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.852149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1159/2298] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:30:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:30:33.859918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:33.860506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:33.860506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:33.883755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] MFA Rules not processed for user `add72a4c345b4ab49e822f478eebd6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:33.885078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:33.955810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:33.998583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1140/2299] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:34.013009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7707e-2dc0-486b-af89-1c6ba876f0e7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.014778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4c7707e-2dc0-486b-af89-1c6ba876f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:34.033970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7707e-2dc0-486b-af89-1c6ba876f0e7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.043089 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.043767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.044627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.044941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.058921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.065054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:34.082800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1160/2300] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:34.843932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:34.987430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156f149-cc53-4c2c-8c45-a3f5ecd8c709 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:34.994818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:34.995789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:34.996102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:34.996384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] PATH_INFO: `/v3/users/1a639e081ed64941aa3e440ad9ffbb7c` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:34.997607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a639e081ed64941aa3e440ad9ffbb7c)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:34.999557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.097174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a639e081ed64941aa3e440ad9ffbb7c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.107577 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a639e081ed64941aa3e440ad9ffbb7c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.126199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a639e081ed64941aa3e440ad9ffbb7c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.126940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a639e081ed64941aa3e440ad9ffbb7c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.139673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a639e081ed64941aa3e440ad9ffbb7c'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.156913 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1141/2301] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:34 2022] DELETE /identity/v3/users/1a639e081ed64941aa3e440ad9ffbb7c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:35.440809 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.478958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6159dcf-2ba9-44a9-b4fa-6bdd4b6e9d99 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.598949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.600133 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.601305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.602016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] PATH_INFO: `/v3/projects/88043393fe3f42bb9e89bb920ded5cb0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.606620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88043393fe3f42bb9e89bb920ded5cb0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.608687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.710444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88043393fe3f42bb9e89bb920ded5cb0', 'request_id': 'req-e0fd9636-de00-48fa-8749-32a37c97d268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.717367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88043393fe3f42bb9e89bb920ded5cb0', 'request_id': 'req-e0fd9636-de00-48fa-8749-32a37c97d268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.726857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88043393fe3f42bb9e89bb920ded5cb0', 'request_id': 'req-e0fd9636-de00-48fa-8749-32a37c97d268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.743237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] The token cache is being invalidate because project 88043393fe3f42bb9e89bb920ded5cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:35.743554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.744219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.746036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1161/2302] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:35 2022] DELETE /identity/v3/projects/88043393fe3f42bb9e89bb920ded5cb0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:35.750484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:35.752245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:35.770526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765b43c-dbf2-4dea-b039-f8e85d167c4a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:35.847437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:35.848941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:35.849302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:35.849650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] PATH_INFO: `/v3/users/8f3ed63aee294774a88636c9e392e29d` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:35.851009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3ed63aee294774a88636c9e392e29d)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:35.853106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:35.856259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:35.960303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3ed63aee294774a88636c9e392e29d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:35.977772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3ed63aee294774a88636c9e392e29d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.009195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3ed63aee294774a88636c9e392e29d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.011782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3ed63aee294774a88636c9e392e29d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.023255 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c88a2309-34ad-49e9-acab-75845f974ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3ed63aee294774a88636c9e392e29d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.030017 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1142/2303] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:35 2022] DELETE /identity/v3/users/8f3ed63aee294774a88636c9e392e29d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:36.310822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.312637 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:36.340399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9636-de00-48fa-8749-32a37c97d268 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:36.437419 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:36.438280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.438572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.439042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] PATH_INFO: `/v3/projects/cfd23bc3225942f9b8e8b021d46c572c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.440262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd23bc3225942f9b8e8b021d46c572c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:36.442471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:36.460320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:36.502917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd23bc3225942f9b8e8b021d46c572c', 'request_id': 'req-c7867e95-a0e3-4e4c-8091-13aa1401e684'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.514867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd23bc3225942f9b8e8b021d46c572c', 'request_id': 'req-c7867e95-a0e3-4e4c-8091-13aa1401e684'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.524471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd23bc3225942f9b8e8b021d46c572c', 'request_id': 'req-c7867e95-a0e3-4e4c-8091-13aa1401e684'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.539573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] The token cache is being invalidate because project cfd23bc3225942f9b8e8b021d46c572c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:36.539873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.540531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c7867e95-a0e3-4e4c-8091-13aa1401e684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:36.542198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1162/2304] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:36 2022] DELETE /identity/v3/projects/cfd23bc3225942f9b8e8b021d46c572c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:36.547473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.547959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.548087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.574208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:36.575262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:36.692048 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1143/2305] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:36.703746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.711811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.711811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.775888 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:36.778034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:36.804180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1163/2306] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:36.813698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.825547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea24a5-e433-40ad-94d1-b5100821aab1 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:36.830597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:36.832292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.832835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.833719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.834652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:36.835131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:36.837068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:36.857321 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1144/2307] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:36.861328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.885602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3d4f8-1457-44a8-ae10-0310398996c9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:36.903063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:36.908529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:36.909126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:36.909361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:36.918056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:36.920291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:36.955794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1164/2308] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:36 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:36.960192 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:36.974547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeaed3-aa68-48cd-b1bb-e29afd58e420 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.068350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.070159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.070462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.070734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.071437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.073617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.091380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:37.104739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1145/2309] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:36 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:37.108698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.121790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55999ed8-022d-4b7a-9cea-1a2cc087e292 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.127421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.128241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.128679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.129338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.130588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.132430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.148341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1165/2310] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:37.155181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.170612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850ffdd-70f8-46aa-86ba-896ff6dde05e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.178213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4fe3a-9fad-4f77-9b20-7315a2fff075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.179153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4fe3a-9fad-4f77-9b20-7315a2fff075 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.179647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4fe3a-9fad-4f77-9b20-7315a2fff075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.180176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4fe3a-9fad-4f77-9b20-7315a2fff075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.181181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4fe3a-9fad-4f77-9b20-7315a2fff075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.183149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4fe3a-9fad-4f77-9b20-7315a2fff075 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.197985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1146/2311] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:37.202021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.243267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755a7ce-0f90-4c8f-94a1-b3d4367f1b36 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.265819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.266725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.267015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.267341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] PATH_INFO: `/v3/projects/a3ff6d0ad68849dfbec2aa2bc0133447/users/4b39883531ac434fb248cd82c60a9404/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.320787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ff6d0ad68849dfbec2aa2bc0133447, user_id=4b39883531ac434fb248cd82c60a9404, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.326661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.350948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1166/2312] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:37 2022] PUT /identity/v3/projects/a3ff6d0ad68849dfbec2aa2bc0133447/users/4b39883531ac434fb248cd82c60a9404/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:37.361054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.361590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.362036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.428390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] MFA Rules not processed for user `4b39883531ac434fb248cd82c60a9404`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:37.431983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:37.547774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1147/2313] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:37.553625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.635618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5a164-fc1d-4ea8-91b5-07dda619e093 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.640474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.641164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.641452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.641728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.642428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.645209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.688602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1167/2314] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:37.692919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.705847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b3962-2573-4520-95d0-3c37d299eabb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.788714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.789642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.790038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.790423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.791241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.793617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.824853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1148/2315] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:37 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:37.826212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.838195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13939b3-dc9c-4b17-8c30-73ae0c613f6d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.842920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.843587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.843891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.844149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.844891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.846509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.858011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1168/2316] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:37.862036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.872514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3f028-e77e-4eac-b270-be9cfd158541 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.880495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:37.881444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:37.881732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:37.882018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] PATH_INFO: `/v3/projects/173f7ac6407f4a0bbd71a54322251996/users/1fac2356d2b1456db0895c67524ec306/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:37.901326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173f7ac6407f4a0bbd71a54322251996, user_id=1fac2356d2b1456db0895c67524ec306, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:37.904151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:37.928378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1149/2317] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:37 2022] PUT /identity/v3/projects/173f7ac6407f4a0bbd71a54322251996/users/1fac2356d2b1456db0895c67524ec306/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:37.937425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:37.962474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39aa00-2ef1-4a1b-9c4a-5ef51acc3bab admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:37.980557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:38.126962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc9730-f87a-4264-97d9-994b0dc78f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.129454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc9730-f87a-4264-97d9-994b0dc78f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.130572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc9730-f87a-4264-97d9-994b0dc78f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.131189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc9730-f87a-4264-97d9-994b0dc78f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.137165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc9730-f87a-4264-97d9-994b0dc78f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.137165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc9730-f87a-4264-97d9-994b0dc78f53 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.152920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1169/2318] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:38.155142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.167230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7b65f-c933-44f4-b66f-116f27ac7d29 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:38.173178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.174189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.174537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.175004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] PATH_INFO: `/v3/projects/173f7ac6407f4a0bbd71a54322251996/users/1fac2356d2b1456db0895c67524ec306/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.188363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173f7ac6407f4a0bbd71a54322251996, user_id=1fac2356d2b1456db0895c67524ec306, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.193256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.208260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1150/2319] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:38 2022] PUT /identity/v3/projects/173f7ac6407f4a0bbd71a54322251996/users/1fac2356d2b1456db0895c67524ec306/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:38.220346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.220573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.220857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.250964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] MFA Rules not processed for user `1fac2356d2b1456db0895c67524ec306`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:38.252328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:38.334245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1170/2320] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:38.345058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:38.346732 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:38.381905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:38.411778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25297820-64e7-4184-9457-6063b1d11742 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:38.415653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd917d09-813e-440f-b96e-c3eb5fce189f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:38.416370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917d09-813e-440f-b96e-c3eb5fce189f service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:38.416679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917d09-813e-440f-b96e-c3eb5fce189f service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:38.417030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd917d09-813e-440f-b96e-c3eb5fce189f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:38.484689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd917d09-813e-440f-b96e-c3eb5fce189f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:38.486549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd917d09-813e-440f-b96e-c3eb5fce189f service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:38.497971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1151/2321] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:41.156669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.220368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a47f4-08bc-4ea4-b846-643b6054e9f9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.226341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.227238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.227520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.227791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] PATH_INFO: `/v3/users/4b39883531ac434fb248cd82c60a9404` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.229016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b39883531ac434fb248cd82c60a9404)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.231004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.277196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:41.288289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b39883531ac434fb248cd82c60a9404'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.295201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b39883531ac434fb248cd82c60a9404'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.307442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b39883531ac434fb248cd82c60a9404'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.308122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b39883531ac434fb248cd82c60a9404'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.316780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b39883531ac434fb248cd82c60a9404'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.323099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1171/2322] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:41 2022] DELETE /identity/v3/users/4b39883531ac434fb248cd82c60a9404 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:41.327685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.328266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.328266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.345710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:41.346933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.414632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1152/2323] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:41.425095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.426939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.498548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257b479-3d5b-4fe2-9fec-d016407987c3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.502296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.502980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.503263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.504128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.514060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.515527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.525861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1172/2324] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:41.873942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:41.875384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:41.905997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a1235-7b4b-4204-b5e2-860b22fb916d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:41.914940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:41.915843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:41.916127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:41.916405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] PATH_INFO: `/v3/projects/a3ff6d0ad68849dfbec2aa2bc0133447` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:41.917859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ff6d0ad68849dfbec2aa2bc0133447)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:41.919800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:41.985018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ff6d0ad68849dfbec2aa2bc0133447', 'request_id': 'req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.991455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ff6d0ad68849dfbec2aa2bc0133447', 'request_id': 'req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:41.999230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ff6d0ad68849dfbec2aa2bc0133447', 'request_id': 'req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.013365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] The token cache is being invalidate because project a3ff6d0ad68849dfbec2aa2bc0133447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:42.013659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.014325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.015901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1153/2325] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:41 2022] DELETE /identity/v3/projects/a3ff6d0ad68849dfbec2aa2bc0133447 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:42.020028 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:42.030391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094604b-e538-49a5-8059-a298368b09aa service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:42.090533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:42.091334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.092102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.092102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] PATH_INFO: `/v3/users/1fac2356d2b1456db0895c67524ec306` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.093148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fac2356d2b1456db0895c67524ec306)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:42.095100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:42.178303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fac2356d2b1456db0895c67524ec306'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.186768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fac2356d2b1456db0895c67524ec306'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.188007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:42.204072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fac2356d2b1456db0895c67524ec306'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.204849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fac2356d2b1456db0895c67524ec306'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.215305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bfe2dff3-2f1d-4bda-a6d9-b6ed31a343dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fac2356d2b1456db0895c67524ec306'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.224371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1173/2326] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:42 2022] DELETE /identity/v3/users/1fac2356d2b1456db0895c67524ec306 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:42.537051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:42.538569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:42.552908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ea2e4-9cf1-4f0f-8359-b5456cfa82ae admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:42.556976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:42.622869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:42.623796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.624085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.624364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] PATH_INFO: `/v3/projects/173f7ac6407f4a0bbd71a54322251996` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.625593 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=173f7ac6407f4a0bbd71a54322251996)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:42.627644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:42.684103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173f7ac6407f4a0bbd71a54322251996', 'request_id': 'req-5eee2db1-a69d-4f3a-a51f-bac0129d6144'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.705904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173f7ac6407f4a0bbd71a54322251996', 'request_id': 'req-5eee2db1-a69d-4f3a-a51f-bac0129d6144'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.715646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173f7ac6407f4a0bbd71a54322251996', 'request_id': 'req-5eee2db1-a69d-4f3a-a51f-bac0129d6144'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.728153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] The token cache is being invalidate because project 173f7ac6407f4a0bbd71a54322251996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:42.728554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.729290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-5eee2db1-a69d-4f3a-a51f-bac0129d6144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:42.730921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1154/2327] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:42 2022] DELETE /identity/v3/projects/173f7ac6407f4a0bbd71a54322251996 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:42.736065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.736466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.736831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.759368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:42.760597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:42.844685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1174/2328] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:42.853598 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.854268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.854682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.877287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:42.878517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:42.887609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1155/2329] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:42.893445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:42.904615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4592ddf-9bb4-4a2d-b6be-b6e5257f9ea1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:42.910333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:42.911439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.912173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.912728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.913782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:42.914635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:42.916767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:42.936979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1175/2330] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:42.939093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:42.949196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb5892-13d5-4719-97cf-7382d3a3767d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:42.953986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:42.954654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:42.955088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:42.955370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:42.956059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:42.958056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:42.986966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1156/2331] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:42.992547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.006300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0628954-18b4-41b8-9e52-824a37a28b1d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.069379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.070053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.070338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.070617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.071304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.073256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.100339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1176/2332] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:42 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:43.102265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.114190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b81d46-249c-4067-9958-b3e60e25aaca admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.119164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.119842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.120161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.120419 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.121182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.122845 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.136406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1157/2333] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:43.140387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.155004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c98b03-9839-4976-b4a3-147d412e7b26 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.159713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6f3bc-48e5-4730-b446-6c7b3c9c35a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.160736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f3bc-48e5-4730-b446-6c7b3c9c35a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.161046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f3bc-48e5-4730-b446-6c7b3c9c35a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.161326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f3bc-48e5-4730-b446-6c7b3c9c35a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.162053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6f3bc-48e5-4730-b446-6c7b3c9c35a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.163669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6f3bc-48e5-4730-b446-6c7b3c9c35a1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.178363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1177/2334] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:43.183295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.198240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d18104-4c6a-4658-b5a6-20f9f055baed admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.212368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.213364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.213647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.213925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] PATH_INFO: `/v3/projects/4e49bd1d522e416fa2f9623e33371f97/users/63adbe79929d4a2ebf0b02547cd631ac/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.261409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e49bd1d522e416fa2f9623e33371f97, user_id=63adbe79929d4a2ebf0b02547cd631ac, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.279632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.297616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1158/2335] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:43 2022] PUT /identity/v3/projects/4e49bd1d522e416fa2f9623e33371f97/users/63adbe79929d4a2ebf0b02547cd631ac/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:43.309750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.310049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.310379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.361795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] MFA Rules not processed for user `63adbe79929d4a2ebf0b02547cd631ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:43.363680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:43.401545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:43.471578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1178/2336] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:43.483118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:43.485120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:43.551363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:43.579392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde67d3-e30e-498e-8e00-4f9351cfae06 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:43.584971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c177d5-eec9-49f2-baaa-43d5e1bd552b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:43.585754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c177d5-eec9-49f2-baaa-43d5e1bd552b service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:43.586122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c177d5-eec9-49f2-baaa-43d5e1bd552b service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:43.586470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c177d5-eec9-49f2-baaa-43d5e1bd552b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:43.595887 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c177d5-eec9-49f2-baaa-43d5e1bd552b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:43.598167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c177d5-eec9-49f2-baaa-43d5e1bd552b service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:43.609811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1159/2337] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:46.686767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:46.778727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cfc61-1a8b-4f23-beb8-8157e2ff335a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:46.787075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:46.788341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:46.788546 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:46.788828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] PATH_INFO: `/v3/users/63adbe79929d4a2ebf0b02547cd631ac` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:46.790881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63adbe79929d4a2ebf0b02547cd631ac)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:46.792788 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:46.860798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63adbe79929d4a2ebf0b02547cd631ac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.875882 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63adbe79929d4a2ebf0b02547cd631ac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.899226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63adbe79929d4a2ebf0b02547cd631ac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.900349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63adbe79929d4a2ebf0b02547cd631ac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.911151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63adbe79929d4a2ebf0b02547cd631ac'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:46.931558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1179/2338] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:30:46 2022] DELETE /identity/v3/users/63adbe79929d4a2ebf0b02547cd631ac => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:46.935983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:46.936508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:46.936594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:46.981235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:46.982955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.059738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1160/2339] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:47.070043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.072390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.144155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f67b0-9451-4374-9584-e1c12a849ed6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.145047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:47.162966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1421b-361f-49b3-ac31-8704d8f73135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.164540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1421b-361f-49b3-ac31-8704d8f73135 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.165471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1421b-361f-49b3-ac31-8704d8f73135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.166037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1421b-361f-49b3-ac31-8704d8f73135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.188003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1421b-361f-49b3-ac31-8704d8f73135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.190285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1421b-361f-49b3-ac31-8704d8f73135 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.201689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1180/2340] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:47.482224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.484202 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.501009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d324c-69f8-4437-89fc-82b2ab7db6d4 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.506509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.507507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.507782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.508102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] PATH_INFO: `/v3/projects/4e49bd1d522e416fa2f9623e33371f97` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.509291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e49bd1d522e416fa2f9623e33371f97)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.511201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.559443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e49bd1d522e416fa2f9623e33371f97', 'request_id': 'req-a48050bb-06fa-4662-8c7a-3673a2357e2d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.564868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e49bd1d522e416fa2f9623e33371f97', 'request_id': 'req-a48050bb-06fa-4662-8c7a-3673a2357e2d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.573470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e49bd1d522e416fa2f9623e33371f97', 'request_id': 'req-a48050bb-06fa-4662-8c7a-3673a2357e2d'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.585328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] The token cache is being invalidate because project 4e49bd1d522e416fa2f9623e33371f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:30:47.585618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.586262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a48050bb-06fa-4662-8c7a-3673a2357e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:30:47.587934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1161/2341] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:30:47 2022] DELETE /identity/v3/projects/4e49bd1d522e416fa2f9623e33371f97 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:30:47.593026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.593322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.593605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.611308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:47.612520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.685262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1181/2342] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:47.693212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.693831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.694186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.715375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:47.716668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:47.725632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1162/2343] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:47.730688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.747816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca83a4-193e-4603-b8e7-644def10e99b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.753155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.753867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.754156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.754440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.755216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:30:47.755528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.757496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.775594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1182/2344] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:30:47.776708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.788391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649de04-5c0a-428a-8c90-7ec42319e05e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.795325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.796456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.797187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.797604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.798431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.800621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.837372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.841009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1163/2345] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:30:47 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:30:47.848572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4f5c8-551f-4fa6-8c99-aa91be33f3ce admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.915779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.916592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.917022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.917422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.918381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.920632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.948786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1183/2346] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:30:47 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:30:47.952688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:47.968763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a545de-0401-4ead-8933-fa11d7f94e98 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:47.973871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:47.975637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:47.976108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:47.976536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:47.977453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:47.979281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:47.994701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1164/2347] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:30:47.998827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:48.009756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ecefc-dada-4db0-b681-0f79ea15da95 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:48.016232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc94cb-9315-4455-940e-b77aff729831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:48.016638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc94cb-9315-4455-940e-b77aff729831 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:48.017667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc94cb-9315-4455-940e-b77aff729831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:48.017967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc94cb-9315-4455-940e-b77aff729831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:48.018716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc94cb-9315-4455-940e-b77aff729831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:48.020351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc94cb-9315-4455-940e-b77aff729831 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:48.034568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1184/2348] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:30:48.040377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:48.059793 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9f18f-70ea-46b2-967d-3e8a2adb0df5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:48.073259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:48.074171 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:48.074653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:48.075085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] PATH_INFO: `/v3/projects/b1f129d1e4d4431c928dc63146543497/users/6eb501c4fd084c619501f565dc16277c/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:48.117190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f129d1e4d4431c928dc63146543497, user_id=6eb501c4fd084c619501f565dc16277c, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:48.121581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:48.125871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:48.147331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1165/2349] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:30:48 2022] PUT /identity/v3/projects/b1f129d1e4d4431c928dc63146543497/users/6eb501c4fd084c619501f565dc16277c/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:30:48.153385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4583a-dcf6-4d86-8a22-76fd470c6f93 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:48.153684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4583a-dcf6-4d86-8a22-76fd470c6f93 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:48.153971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4583a-dcf6-4d86-8a22-76fd470c6f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:48.178399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c6d4583a-dcf6-4d86-8a22-76fd470c6f93 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:48.180346 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c6d4583a-dcf6-4d86-8a22-76fd470c6f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:48.259984 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:48.273582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1185/2350] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:48.306923 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:48.308756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:48.374112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3846-1714-4ec6-8ea6-6a133d842dd4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:48.378262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:48.378920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:48.379210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:48.379492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:48.391394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:48.393451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:48.413154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1166/2351] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:49.689264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356701e-5f7b-47bb-aa08-e098e6de56dd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.689723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356701e-5f7b-47bb-aa08-e098e6de56dd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.689938 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356701e-5f7b-47bb-aa08-e098e6de56dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.719109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7356701e-5f7b-47bb-aa08-e098e6de56dd None None] MFA Rules not processed for user `6eb501c4fd084c619501f565dc16277c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:49.720252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7356701e-5f7b-47bb-aa08-e098e6de56dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:49.792263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1186/2352] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:30:49.804375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:49.805855 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:49.866389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85377-6426-456a-83d7-55c72e84e766 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:49.870018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:49.870677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:49.870961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:49.871239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:49.880016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:49.882172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:49.891415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1167/2353] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:30:51.309591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d81de-b09e-44d7-801a-57f6ee0de818 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:51.310143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d81de-b09e-44d7-801a-57f6ee0de818 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:51.310236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d81de-b09e-44d7-801a-57f6ee0de818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:51.329816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b29d81de-b09e-44d7-801a-57f6ee0de818 None None] MFA Rules not processed for user `6eb501c4fd084c619501f565dc16277c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:51.331143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b29d81de-b09e-44d7-801a-57f6ee0de818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:51.338633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1187/2354] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:30:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:59.887223 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=54 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:01.194185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:01.195646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:01.211501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642744a8-f03b-476e-b3c2-24012db34f88 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:01.217253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:01.218310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:01.218849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:01.219136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] PATH_INFO: `/v3/users/396d8057322842a094fddc09079f5925` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:01.220379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=396d8057322842a094fddc09079f5925)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:01.222357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:01.276283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396d8057322842a094fddc09079f5925'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.284560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396d8057322842a094fddc09079f5925'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.298057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396d8057322842a094fddc09079f5925'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.298784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396d8057322842a094fddc09079f5925'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.307149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396d8057322842a094fddc09079f5925'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.314110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1168/2355] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:31:01 2022] DELETE /identity/v3/users/396d8057322842a094fddc09079f5925 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:01.319308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:01.319799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:01.319905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:01.338901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:01.340097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:01.422633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1188/2356] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:01.432389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:01.434086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:01.505556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988dcb5-b4f3-4e62-9401-6cd520a1bf36 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:01.509576 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd89686-6083-4c7d-a248-2430914d7711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:01.510232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd89686-6083-4c7d-a248-2430914d7711 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:01.510523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd89686-6083-4c7d-a248-2430914d7711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:01.510808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd89686-6083-4c7d-a248-2430914d7711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:01.520435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd89686-6083-4c7d-a248-2430914d7711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:01.522616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd89686-6083-4c7d-a248-2430914d7711 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:01.532665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1169/2357] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:01.802462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:01.804077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:01.818030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556c2-bbf7-4284-8e86-b06b7c268df2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:01.823644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:01.824558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:01.824844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:01.825148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] PATH_INFO: `/v3/projects/d59cb81c13924c599ae44d351268a960` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:01.826508 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59cb81c13924c599ae44d351268a960)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:01.828289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:01.874547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59cb81c13924c599ae44d351268a960', 'request_id': 'req-87f1d5fb-93d7-41e1-aca4-6377a891a179'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.880330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59cb81c13924c599ae44d351268a960', 'request_id': 'req-87f1d5fb-93d7-41e1-aca4-6377a891a179'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.888312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59cb81c13924c599ae44d351268a960', 'request_id': 'req-87f1d5fb-93d7-41e1-aca4-6377a891a179'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.900500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] The token cache is being invalidate because project d59cb81c13924c599ae44d351268a960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:31:01.900780 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.901448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-87f1d5fb-93d7-41e1-aca4-6377a891a179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:31:01.903086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1189/2358] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:31:01 2022] DELETE /identity/v3/projects/d59cb81c13924c599ae44d351268a960 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:01.908233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:01.908557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:01.908860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:01.926929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:01.928301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:01.989780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1170/2359] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:01.998679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:01.998996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:01.999273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.008091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:02.020036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:02.021763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:02.032833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1190/2360] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:02.035818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.045719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258f955-c9fd-4612-8500-f2beda3b3a28 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.050337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.050991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.051280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.051555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.052298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:31:02.052589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.054749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.066307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1171/2361] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:31:02.071304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.080577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2b19f-d766-4e99-8f3f-b07d649cb35f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.085597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.086249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.086539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.086911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.087610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.089542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.114626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1191/2362] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:31:02 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:31:02.119401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.124522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:02.132238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cbe8f-9665-47cc-963e-9872078ca91e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.188685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.189382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.189669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.190014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.190628 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.192543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.213298 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1172/2363] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:31:02 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:31:02.217499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.228013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df73cb1-1be8-4478-b831-18db8c0a3e2b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.233131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.233791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.234078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.234359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.235166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.236714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.247978 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1192/2364] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:31:02.252028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.262870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446dcde-f55e-4491-a027-309a897e48d2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.268147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4ec4f-2d47-44fd-8d4f-8b768e91d111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.268794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4ec4f-2d47-44fd-8d4f-8b768e91d111 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.269114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4ec4f-2d47-44fd-8d4f-8b768e91d111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.269390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4ec4f-2d47-44fd-8d4f-8b768e91d111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.270128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4ec4f-2d47-44fd-8d4f-8b768e91d111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.271837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4ec4f-2d47-44fd-8d4f-8b768e91d111 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.282605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1173/2365] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:31:02.287104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.297159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069705d8-ad20-45a6-9a8a-906fad9019f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.301957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.302874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.303661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.303941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] PATH_INFO: `/v3/projects/0df7a37fa7594d20a4dbb2fe749f407f/users/24661f5180e44accbf410d3a5b355971/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.319545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df7a37fa7594d20a4dbb2fe749f407f, user_id=24661f5180e44accbf410d3a5b355971, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.322184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.334965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1193/2366] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:31:02 2022] PUT /identity/v3/projects/0df7a37fa7594d20a4dbb2fe749f407f/users/24661f5180e44accbf410d3a5b355971/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:31:02.345330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.345633 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.345918 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.375055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] MFA Rules not processed for user `24661f5180e44accbf410d3a5b355971`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:02.376476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:02.459175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1174/2367] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:31:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:02.470717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.473240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:02.534826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993679c6-c506-4f7d-92f5-3a8cb9ec720b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.540212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f356d-2d4c-4091-af67-152a9c87da72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.540917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f356d-2d4c-4091-af67-152a9c87da72 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.541209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f356d-2d4c-4091-af67-152a9c87da72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.541493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f356d-2d4c-4091-af67-152a9c87da72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.550461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f356d-2d4c-4091-af67-152a9c87da72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.551886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f356d-2d4c-4091-af67-152a9c87da72 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.562707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1194/2368] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:08.656123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:08.657702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:08.755091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4cd67-40f5-44de-911d-24a6ec563c59 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:08.760741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:08.761831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:08.761831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:08.762181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:08.764509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:08.777132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:08.778867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:08.785358 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:08.799689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1175/2369] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:12.558214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:28.939075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e8c0-06c1-421b-a91a-69a994be4342 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:28.939724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e8c0-06c1-421b-a91a-69a994be4342 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:28.939724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e8c0-06c1-421b-a91a-69a994be4342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:28.967437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-11c2e8c0-06c1-421b-a91a-69a994be4342 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:28.968694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11c2e8c0-06c1-421b-a91a-69a994be4342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:29.047018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1195/2370] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:29.083428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:29.085080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:29.178180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ea321-b8e8-4007-a396-cf629d6a7966 service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:29.184567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:29.185714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:29.186160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:29.186509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:29.203203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:29.206350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:29.221652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1176/2371] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:31:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:31:37.811405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26feb96f-9455-4fcd-a638-04dc7d80acc0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:37.811405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26feb96f-9455-4fcd-a638-04dc7d80acc0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:37.811405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26feb96f-9455-4fcd-a638-04dc7d80acc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:37.830732 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-26feb96f-9455-4fcd-a638-04dc7d80acc0 None None] MFA Rules not processed for user `347106fdd7c94624ab9d40b51da78623`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:37.832103 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-26feb96f-9455-4fcd-a638-04dc7d80acc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:37.913224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1196/2372] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:31:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:39.216534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=32 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:41.985866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:41.987638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:42.051608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48e07-503c-4624-a36c-c743c6a9bd14 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:42.055811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:42.056463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:42.056752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:42.057088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:42.058925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:42.129727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:42.131256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc3b66-3ac5-40c4-9faa-fd8ed0aaa8a0 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:42.142890 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1177/2373] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:47.893789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=50 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:52.060321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e56f5-6d77-40af-a806-56dc6c9d4e11 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.061401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e56f5-6d77-40af-a806-56dc6c9d4e11 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.061745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e56f5-6d77-40af-a806-56dc6c9d4e11 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.063263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1197/2374] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:31:52.070596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6348e-afb5-4ac0-a4d2-60f16ca07bde None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.071132 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6348e-afb5-4ac0-a4d2-60f16ca07bde None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.071451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6348e-afb5-4ac0-a4d2-60f16ca07bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.089850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-42f6348e-afb5-4ac0-a4d2-60f16ca07bde None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:52.091267 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-42f6348e-afb5-4ac0-a4d2-60f16ca07bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:52.171391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1178/2375] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:52.187132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e56f5-6d77-40af-a806-56dc6c9d4e11 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:52.189126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-952e56f5-6d77-40af-a806-56dc6c9d4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:52.256414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e56f5-6d77-40af-a806-56dc6c9d4e11 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:52.261457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be7ff8-dd8a-4431-8b06-98a6c8194e1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:52.262121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be7ff8-dd8a-4431-8b06-98a6c8194e1c service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.262414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be7ff8-dd8a-4431-8b06-98a6c8194e1c service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.262700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be7ff8-dd8a-4431-8b06-98a6c8194e1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.285877 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be7ff8-dd8a-4431-8b06-98a6c8194e1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:52.289535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be7ff8-dd8a-4431-8b06-98a6c8194e1c service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:52.304486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1198/2376] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:31:52.389042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1397a-df6c-432f-b54e-ef6beb06264b None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.389460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1397a-df6c-432f-b54e-ef6beb06264b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.389823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1397a-df6c-432f-b54e-ef6beb06264b None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.391297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1179/2377] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:31:52.396417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9468a2-f28d-4259-b8be-a9a83804381d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.396773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9468a2-f28d-4259-b8be-a9a83804381d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.397118 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9468a2-f28d-4259-b8be-a9a83804381d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.418341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-db9468a2-f28d-4259-b8be-a9a83804381d None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:52.419482 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-db9468a2-f28d-4259-b8be-a9a83804381d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:52.426123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1199/2378] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:52.439053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1397a-df6c-432f-b54e-ef6beb06264b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:52.452484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1397a-df6c-432f-b54e-ef6beb06264b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:52.456320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377a65f-b735-4e88-b3d7-dc983ebde2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:52.456996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a65f-b735-4e88-b3d7-dc983ebde2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.457284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a65f-b735-4e88-b3d7-dc983ebde2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.457562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a65f-b735-4e88-b3d7-dc983ebde2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.466525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377a65f-b735-4e88-b3d7-dc983ebde2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:52.467980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377a65f-b735-4e88-b3d7-dc983ebde2c7 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:52.478354 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1180/2379] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:52.609425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1413482-6fb3-4f11-8e01-d6b47a046595 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.609819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1413482-6fb3-4f11-8e01-d6b47a046595 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.610177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1413482-6fb3-4f11-8e01-d6b47a046595 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.611678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1200/2380] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:31:52.617718 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.618144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.618525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.639532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:31:52.640743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:52.651368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1181/2381] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:31:52.661849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1413482-6fb3-4f11-8e01-d6b47a046595 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:52.674044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1413482-6fb3-4f11-8e01-d6b47a046595 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:52.678326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7054bdc-0567-46c5-b377-5663bbde730d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:52.678987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7054bdc-0567-46c5-b377-5663bbde730d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:52.679828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7054bdc-0567-46c5-b377-5663bbde730d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:52.680112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7054bdc-0567-46c5-b377-5663bbde730d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:52.693042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7054bdc-0567-46c5-b377-5663bbde730d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:52.694511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7054bdc-0567-46c5-b377-5663bbde730d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:52.707044 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1201/2382] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:32:02.633397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:02.701713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=35 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:05.903289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:05.903894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea7f02-cb88-4b26-8acc-47b26386d96a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:05.904278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea7f02-cb88-4b26-8acc-47b26386d96a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:05.904459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea7f02-cb88-4b26-8acc-47b26386d96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:05.924562 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-84ea7f02-cb88-4b26-8acc-47b26386d96a None None] MFA Rules not processed for user `e73da7d7237d4c9cb11c60a84e4dccd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:05.925488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-84ea7f02-cb88-4b26-8acc-47b26386d96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:05.980393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eff35-57df-4392-8b5d-45fdf545b85d None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:05.985474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:05.986446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:05.986834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:05.987180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] PATH_INFO: `/v3/users/24661f5180e44accbf410d3a5b355971` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:05.988465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1202/2383] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:05.988610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24661f5180e44accbf410d3a5b355971)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:05.990631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:06.062390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24661f5180e44accbf410d3a5b355971'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.076425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24661f5180e44accbf410d3a5b355971'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.090943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24661f5180e44accbf410d3a5b355971'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.091728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24661f5180e44accbf410d3a5b355971'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.101757 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24661f5180e44accbf410d3a5b355971'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.108650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1182/2384] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:32:05 2022] DELETE /identity/v3/users/24661f5180e44accbf410d3a5b355971 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:32:06.113416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.113753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.114013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.133348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:06.134563 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:06.196462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1203/2385] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:32:06.208063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:06.210856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:06.293154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c9ba5-3a12-461b-8bd3-a2c8efd5f6a5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:06.301762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20864034-34ae-4b17-9f04-77b65184f91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:06.302742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864034-34ae-4b17-9f04-77b65184f91d service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.303077 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864034-34ae-4b17-9f04-77b65184f91d service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.303432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864034-34ae-4b17-9f04-77b65184f91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.316945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20864034-34ae-4b17-9f04-77b65184f91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:06.318398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20864034-34ae-4b17-9f04-77b65184f91d service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:06.333323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1183/2386] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:32:06.651479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:06.656261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:06.672107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a896e4-4925-4d20-a82a-61f0d78304cc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:06.680799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:06.681636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.683487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.683641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] PATH_INFO: `/v3/projects/0df7a37fa7594d20a4dbb2fe749f407f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.685202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df7a37fa7594d20a4dbb2fe749f407f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:06.687631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:06.739163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df7a37fa7594d20a4dbb2fe749f407f', 'request_id': 'req-a695c8e9-4d68-4687-8395-2f5a042277b3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.746471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df7a37fa7594d20a4dbb2fe749f407f', 'request_id': 'req-a695c8e9-4d68-4687-8395-2f5a042277b3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.753893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df7a37fa7594d20a4dbb2fe749f407f', 'request_id': 'req-a695c8e9-4d68-4687-8395-2f5a042277b3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.766818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] The token cache is being invalidate because project 0df7a37fa7594d20a4dbb2fe749f407f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:32:06.767426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.768257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a695c8e9-4d68-4687-8395-2f5a042277b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:06.770696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1204/2387] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:32:06 2022] DELETE /identity/v3/projects/0df7a37fa7594d20a4dbb2fe749f407f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:32:06.778957 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdfca4-8b82-41f0-a534-5cda790a31c8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.779484 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdfca4-8b82-41f0-a534-5cda790a31c8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.780154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdfca4-8b82-41f0-a534-5cda790a31c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.817589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-1abdfca4-8b82-41f0-a534-5cda790a31c8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:06.819117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1abdfca4-8b82-41f0-a534-5cda790a31c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:06.849289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:06.907593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.908184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.908493 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.932319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] MFA Rules not processed for user `6eb501c4fd084c619501f565dc16277c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:06.933765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:06.937482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1184/2388] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:06.947625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:06.948169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:06.950091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:06.974684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:07.014043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:07.015341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.040408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1205/2389] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:07.042203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1185/2390] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:07.050929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.094095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9633-8ccc-4774-b4d5-31990e3a30da None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.101375 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.102418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.102774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.103124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.104030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:32:07.104490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.106045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c28f4-4fdb-4a69-84c4-15327c74d1dd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.122031 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1206/2391] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:32:07.126042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.136811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209ca34-c552-4a77-9b68-b6c2380c98e0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.165303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.166014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.166457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.166745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.167466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.169467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.197159 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.197775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.198211 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.208674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1186/2392] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:32:07 2022] POST /identity/v3/projects => generated 391 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:32:07.213126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.223432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:07.225092 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.311218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1207/2393] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:32:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:32:07.326498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.329892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.355197 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2be40e-9ec8-46ed-b7f5-64aa74e14643 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.360146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.360955 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.361251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.361531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.362286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.364257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.400778 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1187/2394] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:32:07 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:32:07.404374 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.418462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4546d-e2be-474d-97c6-3557217183f2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.425175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.425946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.426239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.426574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.427514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.430028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.439858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045e530-d297-4796-8967-d27aa2e53204 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.442181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1188/2395] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:32:07.446543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.446749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.447359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.447725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.448015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.458876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.459570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a61b-8daa-400b-9029-6d988f1201af admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.461384 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.466630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715f93-ba41-4d27-9b8a-edf5de698f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.467426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715f93-ba41-4d27-9b8a-edf5de698f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.467801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715f93-ba41-4d27-9b8a-edf5de698f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.468031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715f93-ba41-4d27-9b8a-edf5de698f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.468779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715f93-ba41-4d27-9b8a-edf5de698f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.470562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715f93-ba41-4d27-9b8a-edf5de698f51 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.471961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1208/2396] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:32:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:32:07.485060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1189/2397] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:32:07.488799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.500668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f4b03-3934-48d3-b1d1-0ee94d23679b service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:07.506255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:07.508137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.508611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.508934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] PATH_INFO: `/v3/projects/668acfcf74d741549a3aeca735638144/users/89102e9db92a4e8a93e2b55f11a1d9f1/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.545339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=668acfcf74d741549a3aeca735638144, user_id=89102e9db92a4e8a93e2b55f11a1d9f1, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:07.549392 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:07.577398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1209/2398] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:32:07 2022] PUT /identity/v3/projects/668acfcf74d741549a3aeca735638144/users/89102e9db92a4e8a93e2b55f11a1d9f1/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:32:07.579411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:07.579546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:07.579756 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:07.653016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] MFA Rules not processed for user `89102e9db92a4e8a93e2b55f11a1d9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:07.654755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.795380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:07.829871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1190/2399] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:07.842429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:07.844453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:07.939803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:08.005379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658a3c-e3b4-469d-9aeb-37674dd518e1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:08.015921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2f918-cd75-4615-972d-83c0df0e677b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:08.017777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2f918-cd75-4615-972d-83c0df0e677b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:08.018511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2f918-cd75-4615-972d-83c0df0e677b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:08.020171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2f918-cd75-4615-972d-83c0df0e677b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:08.045169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2f918-cd75-4615-972d-83c0df0e677b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:08.047144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2f918-cd75-4615-972d-83c0df0e677b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:08.077249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1210/2400] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:32:17.825108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:18.061945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=12 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:27.730058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:27.820911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c210c8f-0b0a-48f1-bb01-c23298290127 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:27.825841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:27.826758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:27.827043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:27.827320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] PATH_INFO: `/v3/users/89102e9db92a4e8a93e2b55f11a1d9f1` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:27.828832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89102e9db92a4e8a93e2b55f11a1d9f1)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:27.831063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:27.890060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89102e9db92a4e8a93e2b55f11a1d9f1'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:27.897989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89102e9db92a4e8a93e2b55f11a1d9f1'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:27.920411 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89102e9db92a4e8a93e2b55f11a1d9f1'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:27.922436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89102e9db92a4e8a93e2b55f11a1d9f1'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:27.934606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89102e9db92a4e8a93e2b55f11a1d9f1'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:27.946881 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1191/2401] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:32:27 2022] DELETE /identity/v3/users/89102e9db92a4e8a93e2b55f11a1d9f1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:32:27.948650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:27.949842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:27.951340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:27.978496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:27.980500 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:28.056124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1211/2402] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:28.062503 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:28.064025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:28.130878 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091b269-0a48-4d54-9f77-79688b9aef1c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:28.135768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc850-7c66-488c-94de-ba8338cedb54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:28.136704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc850-7c66-488c-94de-ba8338cedb54 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:28.137182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc850-7c66-488c-94de-ba8338cedb54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:28.137640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc850-7c66-488c-94de-ba8338cedb54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:28.149739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cc850-7c66-488c-94de-ba8338cedb54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:28.151469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cc850-7c66-488c-94de-ba8338cedb54 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:28.162329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1192/2403] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:32:28.497471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:28.499787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:28.521673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64473-87f7-4c92-bfcf-b93b5a95ca88 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:28.528523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:28.529510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:28.529836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:28.530120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] PATH_INFO: `/v3/projects/668acfcf74d741549a3aeca735638144` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:28.531484 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=668acfcf74d741549a3aeca735638144)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:28.534536 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:28.599349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668acfcf74d741549a3aeca735638144', 'request_id': 'req-af5b1f84-7025-4be2-81b2-5f7d309d77e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:28.607280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668acfcf74d741549a3aeca735638144', 'request_id': 'req-af5b1f84-7025-4be2-81b2-5f7d309d77e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:28.617187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668acfcf74d741549a3aeca735638144', 'request_id': 'req-af5b1f84-7025-4be2-81b2-5f7d309d77e7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:28.629738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] The token cache is being invalidate because project 668acfcf74d741549a3aeca735638144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:32:28.629738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:28.630334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:32:28.632851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1212/2404] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:32:28 2022] DELETE /identity/v3/projects/668acfcf74d741549a3aeca735638144 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:32:33.050041 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454990e-42df-4494-8584-ff44642127e0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:33.050554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454990e-42df-4494-8584-ff44642127e0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:33.050746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454990e-42df-4494-8584-ff44642127e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:33.069018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9454990e-42df-4494-8584-ff44642127e0 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:33.070813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9454990e-42df-4494-8584-ff44642127e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:33.089117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:33.135018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1193/2405] 10.222.0.79 () {58 vars in 967 bytes} [Wed Jul 20 19:32:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:33.157689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:33.159232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:33.230220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b1f84-7025-4be2-81b2-5f7d309d77e7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:33.235719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:33.236474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:33.236766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:33.237077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:33.246577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:33.248113 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:33.257200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1213/2406] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:32:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:32:41.590651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454990e-42df-4494-8584-ff44642127e0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:41.592180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9454990e-42df-4494-8584-ff44642127e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:41.658184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454990e-42df-4494-8584-ff44642127e0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:41.662678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:41.663344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:41.663627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:41.663932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:41.666225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:41.754207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:41.755650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361ca85-2ea7-47c0-9fc1-9e57d521692a service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:41.768008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1194/2407] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:32:41.796465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:41.797986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:41.814466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:41.864033 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a69dc-c717-4554-b31b-a7fbb1c62ba6 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:41.867714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:41.868474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:41.868758 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:41.869078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:41.870905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:41.886354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:41.888048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:41.901649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1214/2408] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:32:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:32:51.762988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=69 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:51.894727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=20 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:32:53.091006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:53.091506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:53.091819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:53.116141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:32:53.118401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:32:53.128280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1195/2409] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:32:53.147309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:32:53.159099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686a387-f34b-4df5-b11e-366f0798dabc service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:32:53.171750 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968a36a-3313-4f68-8ec8-69e65725e8ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:32:53.172651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968a36a-3313-4f68-8ec8-69e65725e8ce service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:32:53.173685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968a36a-3313-4f68-8ec8-69e65725e8ce service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:32:53.173685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968a36a-3313-4f68-8ec8-69e65725e8ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:32:53.185107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968a36a-3313-4f68-8ec8-69e65725e8ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:32:53.186789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968a36a-3313-4f68-8ec8-69e65725e8ce service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:32:53.196681 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1215/2410] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:33:00.740284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:00.742946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:00.762850 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c09b0-af85-4f63-b255-803233c89ce7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:00.769001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:00.769897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:00.770257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:00.770612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:00.772542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:00.855390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:00.857347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:00.872368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1196/2411] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:03.192947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:10.865635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=31 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:11.986023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:11.986492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:11.986586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:12.005429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:12.006665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:12.014237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1216/2412] 10.222.0.79 () {58 vars in 967 bytes} [Wed Jul 20 19:33:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:33:12.027064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:12.039336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28050c9e-49b7-470a-bd44-f9957dee2129 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:12.044464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:12.045162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:12.045452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:12.045734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:12.055186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:12.057019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:12.065648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1197/2413] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:33:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:33:14.082189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:14.083753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:14.158696 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa55bfe-a3da-42c8-821c-c843f2f490e9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:14.163804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:14.165106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.165329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.166366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] PATH_INFO: `/v3/users/6eb501c4fd084c619501f565dc16277c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.167645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb501c4fd084c619501f565dc16277c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:14.169937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:14.229564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb501c4fd084c619501f565dc16277c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.239324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb501c4fd084c619501f565dc16277c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.252637 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb501c4fd084c619501f565dc16277c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.253531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb501c4fd084c619501f565dc16277c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.262414 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-344af1ee-6094-46d0-92b5-5d931253359c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb501c4fd084c619501f565dc16277c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.270078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1217/2414] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:33:14 2022] DELETE /identity/v3/users/6eb501c4fd084c619501f565dc16277c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:14.453610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:14.464748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc80ad2-98b0-4bf1-97b6-a6483f941fc5 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:14.539471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:14.541722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.541722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.541722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] PATH_INFO: `/v3/projects/b1f129d1e4d4431c928dc63146543497` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.542753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f129d1e4d4431c928dc63146543497)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:14.545669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:14.594516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f129d1e4d4431c928dc63146543497', 'request_id': 'req-8c623e94-44e5-4bdb-8290-cd3e58705ba7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.600319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f129d1e4d4431c928dc63146543497', 'request_id': 'req-8c623e94-44e5-4bdb-8290-cd3e58705ba7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.608868 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f129d1e4d4431c928dc63146543497', 'request_id': 'req-8c623e94-44e5-4bdb-8290-cd3e58705ba7'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.621800 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] The token cache is being invalidate because project b1f129d1e4d4431c928dc63146543497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:33:14.622096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.622851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c623e94-44e5-4bdb-8290-cd3e58705ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:14.624495 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1198/2415] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:33:14 2022] DELETE /identity/v3/projects/b1f129d1e4d4431c928dc63146543497 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:14.629154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.629446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.629731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.647819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:14.649859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:14.711403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1218/2416] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:33:14.719105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.719399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.719682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.739789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:14.741095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:14.750002 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1199/2417] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:33:14.755098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:14.765636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebf92d-8c42-4e14-8683-f270a457ba23 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:14.770319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:14.770983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.771268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.771534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.772405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:33:14.772778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:14.775061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:14.784344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:14.792386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1219/2418] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:33:14.796472 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:14.806412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019616ba-9848-4fb1-9a6e-66395dc2ad30 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:14.811440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:14.812103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.812384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.812767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.813505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:14.815639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:14.852775 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1200/2419] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:33:14 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:33:14.856538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:14.867199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9af85f-4f34-4832-a7a0-952b8892bec8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:14.930503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:14.931179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.931495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.931799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.932491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:14.934487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:14.954688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1220/2420] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:33:14 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:33:14.958148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:14.969314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eda672-c8c1-45c9-9942-1d0c42b72c52 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:14.973933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:14.974585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:14.974869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:14.975145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:14.975898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:14.977736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:14.988242 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1201/2421] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:33:14.992467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:15.002686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603de22-ec57-4a45-95c9-0dc11d191f03 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:15.007886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22e16b-0ec0-45bc-8317-b72b0e5d30c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:15.008720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22e16b-0ec0-45bc-8317-b72b0e5d30c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:15.009030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22e16b-0ec0-45bc-8317-b72b0e5d30c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:15.009308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22e16b-0ec0-45bc-8317-b72b0e5d30c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:15.010032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22e16b-0ec0-45bc-8317-b72b0e5d30c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:15.011702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22e16b-0ec0-45bc-8317-b72b0e5d30c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:15.021743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1221/2422] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:33:15.026945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:15.037054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114119-a0da-45c4-b7d2-acbdafcf3d80 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:15.041685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:15.042553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:15.042837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:15.043113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] PATH_INFO: `/v3/projects/cb88098055524c79bdd3b7d70ce980ca/users/873c110b36fc4e6287def2e768448dc3/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:15.058448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb88098055524c79bdd3b7d70ce980ca, user_id=873c110b36fc4e6287def2e768448dc3, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:15.061767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:15.077391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1202/2423] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:33:15 2022] PUT /identity/v3/projects/cb88098055524c79bdd3b7d70ce980ca/users/873c110b36fc4e6287def2e768448dc3/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:15.088552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:15.089099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:15.089244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:15.118965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] MFA Rules not processed for user `873c110b36fc4e6287def2e768448dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:15.120180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:15.193195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1222/2424] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:33:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:33:15.205417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:15.207629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:15.228500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:15.278154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4cf0c-fd09-49e5-9c24-4e075b182a15 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:15.281797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:15.282475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:15.282761 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:15.283040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:15.292049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:15.293559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:15.303771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1203/2425] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:22.120777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:22.122522 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:22.204734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e091e-4ea1-4e8c-b662-4b34a55a98f1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:22.210272 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:22.210918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:22.211821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:22.212177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:22.214214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:22.223243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:22.227279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:22.229246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916c4b0-8ac1-4980-bd3b-f7175f6787dc service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:22.240630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1223/2426] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:25.298731 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=20 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:25.584777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:25.664292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc86-7126-48e4-aa6e-5e95bc882826 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:25.669437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:25.670340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:25.670611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:25.670889 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] PATH_INFO: `/v3/users/873c110b36fc4e6287def2e768448dc3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:25.672088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=873c110b36fc4e6287def2e768448dc3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:25.674065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:25.731056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873c110b36fc4e6287def2e768448dc3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:25.741490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873c110b36fc4e6287def2e768448dc3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:25.755096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873c110b36fc4e6287def2e768448dc3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:25.756000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873c110b36fc4e6287def2e768448dc3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:25.765035 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873c110b36fc4e6287def2e768448dc3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:25.771159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1204/2427] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:33:25 2022] DELETE /identity/v3/users/873c110b36fc4e6287def2e768448dc3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:25.776186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb016a-26d1-4b1e-a8a5-92e9bbf3407e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:25.776694 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb016a-26d1-4b1e-a8a5-92e9bbf3407e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:25.776789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb016a-26d1-4b1e-a8a5-92e9bbf3407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:25.795669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-68bb016a-26d1-4b1e-a8a5-92e9bbf3407e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:25.797180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-68bb016a-26d1-4b1e-a8a5-92e9bbf3407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:25.860674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1224/2428] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:33:25.871186 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:25.873357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:25.943429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab183f-7441-41e8-af28-c362cebb04b0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:25.948837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:25.949551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:25.949823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:25.950168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:25.960117 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:25.962005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:25.972558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1205/2429] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:26.252487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.256229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.256514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.269653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:26.271998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.277156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] MFA Rules not processed for user `493d126879f44b26951920235e7f7730`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:26.278625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.296357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1973b2b-5d04-45fd-baa9-5157d419e007 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:26.301093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:26.301951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.302239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.302516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] PATH_INFO: `/v3/projects/cb88098055524c79bdd3b7d70ce980ca` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.303687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb88098055524c79bdd3b7d70ce980ca)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:26.306701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:26.324785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:26.342199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1225/2430] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:33:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:33:26.362935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb88098055524c79bdd3b7d70ce980ca', 'request_id': 'req-53f42e34-e9f7-4357-a892-a839b86e4bcb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.369573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb88098055524c79bdd3b7d70ce980ca', 'request_id': 'req-53f42e34-e9f7-4357-a892-a839b86e4bcb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.377830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb88098055524c79bdd3b7d70ce980ca', 'request_id': 'req-53f42e34-e9f7-4357-a892-a839b86e4bcb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.391519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] The token cache is being invalidate because project cb88098055524c79bdd3b7d70ce980ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:33:26.391834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.392473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:26.394050 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1206/2431] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:33:26 2022] DELETE /identity/v3/projects/cb88098055524c79bdd3b7d70ce980ca => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:26.683348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:26.684868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.697435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8b9a-1856-40e7-bc5c-e8e773d8001c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:26.749007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:26.749661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:26.749945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:26.750224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:26.752004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:26.761705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:26.763121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:26.770995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1226/2432] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:27.221284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:27.222944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:27.297910 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f42e34-e9f7-4357-a892-a839b86e4bcb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:27.302888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:27.303536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:27.303818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:27.304092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:27.306446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:27.317892 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:27.319465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:27.330191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1207/2433] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:29.547912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:29.550060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:29.562138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7149d-044d-4c3b-9147-f1e8919bb3de service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:29.568205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:29.568866 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:29.569311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:29.569772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:29.572443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:29.585978 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:29.587451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:29.602030 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1227/2434] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:35.660787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:35.662526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:35.746712 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610dcc9-b97e-40a6-9c3b-955da834864a service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:35.752481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:35.753235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:35.753526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:35.753807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:35.755606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:35.788960 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:35.818968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:35.820788 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:35.830605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1208/2435] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:37.558017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:37.572963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27924-d23d-4931-b1ec-a3db0a80ea91 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:37.578895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:37.582571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:37.582571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:37.582571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:37.582571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:37.613505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:37.653361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:37.654955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:37.664573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1228/2436] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:43.945544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:43.948697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:44.090317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4894d6-0137-4a97-9326-1d79b552707e service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:44.096153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:44.097221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:44.097572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:44.097941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] PATH_INFO: `/v3/users/e73da7d7237d4c9cb11c60a84e4dccd8` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:44.099293 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73da7d7237d4c9cb11c60a84e4dccd8)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:44.101930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:44.159213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73da7d7237d4c9cb11c60a84e4dccd8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.166562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73da7d7237d4c9cb11c60a84e4dccd8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.180841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73da7d7237d4c9cb11c60a84e4dccd8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.181639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73da7d7237d4c9cb11c60a84e4dccd8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.191389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73da7d7237d4c9cb11c60a84e4dccd8'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.198456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1209/2437] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:33:43 2022] DELETE /identity/v3/users/e73da7d7237d4c9cb11c60a84e4dccd8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:44.516784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:44.536801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624609d8-14cb-4b40-9de6-755272d805aa service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:44.624279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:44.625444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:44.626950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:44.626950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] PATH_INFO: `/v3/projects/2aa8c8dd804b46ca8996788a32b8383e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:44.629093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa8c8dd804b46ca8996788a32b8383e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:44.631061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:44.702140 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa8c8dd804b46ca8996788a32b8383e', 'request_id': 'req-a22604b4-f214-4775-b56f-1fa48e783149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.711733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa8c8dd804b46ca8996788a32b8383e', 'request_id': 'req-a22604b4-f214-4775-b56f-1fa48e783149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.722577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa8c8dd804b46ca8996788a32b8383e', 'request_id': 'req-a22604b4-f214-4775-b56f-1fa48e783149'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.737705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] The token cache is being invalidate because project 2aa8c8dd804b46ca8996788a32b8383e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:33:44.738635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.739439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a22604b4-f214-4775-b56f-1fa48e783149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:33:44.741473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1229/2438] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:33:44 2022] DELETE /identity/v3/projects/2aa8c8dd804b46ca8996788a32b8383e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:33:52.542448 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:52.544916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:52.605377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:52.636167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a388f-ada7-4491-9814-97352c5c0067 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:52.717652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:52.718482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:52.718876 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:52.719261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:52.721982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:52.735415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:52.737242 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:52.749703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1210/2439] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:54.126486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:54.126997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:54.127110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:54.144726 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] MFA Rules not processed for user `c18c8d4034ce40dba9f9f48e96a19814`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:33:54.146060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:54.207256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1230/2440] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:33:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:33:54.530464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:54.532111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:54.598446 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f40e4-da71-470d-9095-81787b9d34c5 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:54.602119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:54.602787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:54.603056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:54.603332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:54.605142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:54.616830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:54.618547 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debf6c-8355-495f-ad7a-40c8041c9a75 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:54.627914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1211/2441] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:03.327195 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:03.328975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:03.341730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:03.350161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5287f-ae14-4e25-a5cc-21ba5a082ce6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:03.358605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:03.359551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:03.359842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:03.360125 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:03.362356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:03.377192 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:03.378937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c12b5-ba8a-4756-9381-322b47d8fbfd service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:03.392009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1231/2442] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:04.623717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=65 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:07.007799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282902b-c416-457f-af20-ce3a56f45df9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:07.008504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282902b-c416-457f-af20-ce3a56f45df9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:07.008504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282902b-c416-457f-af20-ce3a56f45df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:07.031564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0282902b-c416-457f-af20-ce3a56f45df9 None None] MFA Rules not processed for user `347106fdd7c94624ab9d40b51da78623`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:34:07.033173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0282902b-c416-457f-af20-ce3a56f45df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:07.092370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1212/2443] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:34:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:34:13.387616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:17.084408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=25 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:19.600358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260673e3-567c-495e-859f-cfe404ca282b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:19.600848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260673e3-567c-495e-859f-cfe404ca282b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:19.600982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260673e3-567c-495e-859f-cfe404ca282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:19.619819 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-260673e3-567c-495e-859f-cfe404ca282b None None] MFA Rules not processed for user `3f7758d298f642138b71aa422d0344da`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:34:19.621635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-260673e3-567c-495e-859f-cfe404ca282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:19.677947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1232/2444] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:34:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:34:20.310533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:20.311038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:20.311139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:20.330764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:34:20.331978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:20.391301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1213/2445] 10.222.0.79 () {58 vars in 967 bytes} [Wed Jul 20 19:34:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:34:20.441827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260673e3-567c-495e-859f-cfe404ca282b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:20.443278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-260673e3-567c-495e-859f-cfe404ca282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:20.519496 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260673e3-567c-495e-859f-cfe404ca282b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:20.524842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:20.525559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:20.526012 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:20.526286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:20.536332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:20.538286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:20.548540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1233/2446] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:34:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:34:24.583582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:24.585051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:24.677299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e3caf-9edd-4ade-9f53-165c5d818e79 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:24.682822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:24.684096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:24.684501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:24.685009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:24.687447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:24.702043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:24.703733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:24.714883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1214/2447] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:28.411396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:28.413220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:28.480191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03525157-1d8d-47be-b154-ba131476c033 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:28.484805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:28.485704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:28.485986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:28.486261 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] PATH_INFO: `/v3/users/43929f4dcce44f3bac9fd69cf960c985` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:28.487479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43929f4dcce44f3bac9fd69cf960c985)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:28.489431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:28.530555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:28.540075 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43929f4dcce44f3bac9fd69cf960c985'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.547428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43929f4dcce44f3bac9fd69cf960c985'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.560369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43929f4dcce44f3bac9fd69cf960c985'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.561148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43929f4dcce44f3bac9fd69cf960c985'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.568358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43929f4dcce44f3bac9fd69cf960c985'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.574983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1234/2448] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:34:28 2022] DELETE /identity/v3/users/43929f4dcce44f3bac9fd69cf960c985 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:34:28.754924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:28.768620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83e7f-f06f-4e6f-b86d-4e686bb04196 service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:28.828665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:28.829485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:28.829776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:28.830057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] PATH_INFO: `/v3/projects/91e1220a765f41e88d4e3480b9a7a687` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:28.831254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e1220a765f41e88d4e3480b9a7a687)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:28.833222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:28.873940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:28.892053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e1220a765f41e88d4e3480b9a7a687', 'request_id': 'req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.898642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e1220a765f41e88d4e3480b9a7a687', 'request_id': 'req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.907686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e1220a765f41e88d4e3480b9a7a687', 'request_id': 'req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.921398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] The token cache is being invalidate because project 91e1220a765f41e88d4e3480b9a7a687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:34:28.921953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.922784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:28.924789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1215/2449] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:34:28 2022] DELETE /identity/v3/projects/91e1220a765f41e88d4e3480b9a7a687 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:34:28.928627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:28.930144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:28.949241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a856e3-806a-4860-be76-0256b5d34485 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:29.014297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:29.015389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:29.015502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:29.015831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] PATH_INFO: `/v3/users/3e50a128b14743fa8d8cd38b4201ad6e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:29.016957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e50a128b14743fa8d8cd38b4201ad6e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:29.018918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:29.065447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e50a128b14743fa8d8cd38b4201ad6e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.072531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e50a128b14743fa8d8cd38b4201ad6e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.084636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e50a128b14743fa8d8cd38b4201ad6e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.085278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e50a128b14743fa8d8cd38b4201ad6e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.092982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e45cf4e1-dd0b-4c29-a2cb-286649e105a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e50a128b14743fa8d8cd38b4201ad6e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.098889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1235/2450] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:34:28 2022] DELETE /identity/v3/users/3e50a128b14743fa8d8cd38b4201ad6e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:34:29.275703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:29.277875 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:34:29.295226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942cc1-f301-413a-b2db-72c0b5d7aa5b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:29.354236 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:29.355751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:29.356037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:29.356316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] PATH_INFO: `/v3/projects/2aff2e1ddb6f4c2e9818edc9babdd7f2` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:29.357720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aff2e1ddb6f4c2e9818edc9babdd7f2)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:29.359949 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:29.413380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aff2e1ddb6f4c2e9818edc9babdd7f2', 'request_id': 'req-8846b5a1-a746-4eb7-8219-af11ee601bff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.419659 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aff2e1ddb6f4c2e9818edc9babdd7f2', 'request_id': 'req-8846b5a1-a746-4eb7-8219-af11ee601bff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.427054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aff2e1ddb6f4c2e9818edc9babdd7f2', 'request_id': 'req-8846b5a1-a746-4eb7-8219-af11ee601bff'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.438909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] The token cache is being invalidate because project 2aff2e1ddb6f4c2e9818edc9babdd7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:34:29.439180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.439859 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:34:29.441431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1216/2451] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:34:29 2022] DELETE /identity/v3/projects/2aff2e1ddb6f4c2e9818edc9babdd7f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:34:39.095386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=68,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:39.435991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:23.953399 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:23.954003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:23.954124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:23.972108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:23.973276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:24.039710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1236/2452] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:35:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:24.088539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:24.090158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:24.185905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846b5a1-a746-4eb7-8219-af11ee601bff admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:24.192696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:24.196663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:24.196663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:24.196663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:24.203546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:24.206386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:24.215181 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1217/2453] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:35:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:35:25.766286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:25.773911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:25.859987 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8266ab-78e9-4d6c-92a3-55ca64b06151 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:25.863701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:25.864385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:25.864690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:25.865108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:25.866920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:25.955154 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:25.956977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:25.970198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1237/2454] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:30.065556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:30.067187 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:30.092306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd66c0b-1b09-4d42-a8d2-7ec5cba409d3 service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:30.097273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:30.098195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:30.099120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:30.100288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] PATH_INFO: `/v3/users/347106fdd7c94624ab9d40b51da78623` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:30.100694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=347106fdd7c94624ab9d40b51da78623)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:30.102673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:30.182169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347106fdd7c94624ab9d40b51da78623'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.197538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347106fdd7c94624ab9d40b51da78623'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.224485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347106fdd7c94624ab9d40b51da78623'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.226405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347106fdd7c94624ab9d40b51da78623'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.235883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-48923ccd-b6a1-4887-a758-186a79bc7d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347106fdd7c94624ab9d40b51da78623'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.243188 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1218/2455] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:30 2022] DELETE /identity/v3/users/347106fdd7c94624ab9d40b51da78623 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:30.473474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:30.491104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fa74-59cf-468a-96fd-e44b4fdd2fea service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:30.565333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:30.574068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:30.574952 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:30.575299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:30.575582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] PATH_INFO: `/v3/projects/9c025843a2de49d48c39b40f2261d3bf` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:30.576805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c025843a2de49d48c39b40f2261d3bf)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:30.578856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:30.648786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c025843a2de49d48c39b40f2261d3bf', 'request_id': 'req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.659439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c025843a2de49d48c39b40f2261d3bf', 'request_id': 'req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.668243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c025843a2de49d48c39b40f2261d3bf', 'request_id': 'req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.688771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] The token cache is being invalidate because project 9c025843a2de49d48c39b40f2261d3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:30.688771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.688771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a90a9193-6139-4c79-abaf-ee0de6d1e7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:30.692981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1238/2456] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:30 2022] DELETE /identity/v3/projects/9c025843a2de49d48c39b40f2261d3bf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:30.696080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:30.705529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:30.707058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:30.734625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:30.736699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:30.846135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1219/2457] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:30.853730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:30.854376 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:30.855021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:30.888707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:30.893003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:30.931573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1239/2458] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:30.937726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:30.945075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:30.952543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb0e2-2ac7-4b6e-b522-f111553bbce3 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:30.967744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:30.973843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:30.974378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:30.975018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:30.975970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:35:30.976291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:30.978133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:30.994329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1220/2459] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:35:30.998622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:31.026765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d8ec7-3409-4304-852e-8630356e57a4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:31.032815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:31.034793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.035098 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.035475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.036065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:31.038716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:31.077631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1240/2460] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:30 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:31.082747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:31.094517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a323dd6-e94c-4d5c-898b-3beab36fa256 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:31.170849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:31.171719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.172112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.172539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.173397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:31.175457 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:31.208443 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1221/2461] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:31 2022] POST /identity/v3/users => generated 284 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:31.210885 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:31.227659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b4c6-2140-4c2a-ad03-2c31139a1583 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:31.238057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:31.238854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.239149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.239430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.240232 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:31.241896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:31.264727 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1241/2462] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:31.265947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:31.281586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19a3d-75f6-4ac4-9659-105ac679b9cd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:31.292596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd607822-d411-4d31-82b9-3d366cb2503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:31.293744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607822-d411-4d31-82b9-3d366cb2503d admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.294456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607822-d411-4d31-82b9-3d366cb2503d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.295149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607822-d411-4d31-82b9-3d366cb2503d admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.297572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd607822-d411-4d31-82b9-3d366cb2503d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:31.299307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd607822-d411-4d31-82b9-3d366cb2503d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:31.315741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1222/2463] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:31.317957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:31.329991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d78d6-e4a4-4419-a49c-4d3728921bfe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:31.335996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:31.337042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.337330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.337671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] PATH_INFO: `/v3/projects/96fa88f82fe64faaaa774e8c9c963609/users/c7a9422dd1f348c5a60af03d0128ce59/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.367269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fa88f82fe64faaaa774e8c9c963609, user_id=c7a9422dd1f348c5a60af03d0128ce59, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:31.370179 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:31.389704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1242/2464] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:31 2022] PUT /identity/v3/projects/96fa88f82fe64faaaa774e8c9c963609/users/c7a9422dd1f348c5a60af03d0128ce59/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:35:31.401559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.402128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.402638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.457981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] MFA Rules not processed for user `c7a9422dd1f348c5a60af03d0128ce59`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:31.459206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:31.602310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1223/2465] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:31.617136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:31.619308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:31.743279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb02356-3472-42a8-bc26-004d920072c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:31.747490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe8395-3a2b-4ed6-bfed-bc6b5f5c9f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:31.748138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8395-3a2b-4ed6-bfed-bc6b5f5c9f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:31.748565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8395-3a2b-4ed6-bfed-bc6b5f5c9f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:31.748714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8395-3a2b-4ed6-bfed-bc6b5f5c9f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:31.758333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe8395-3a2b-4ed6-bfed-bc6b5f5c9f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:31.759783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe8395-3a2b-4ed6-bfed-bc6b5f5c9f1a service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:31.774450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1243/2466] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:35:35.331994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:35.341726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d089-2cbd-419f-a721-a98ec6a57571 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:35.346158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:35.346944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:35.347299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:35.347669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:35.349728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:35.410580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:35.413329 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:35.415507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:35.426651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1224/2467] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:41.764851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:42.723682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e544f-b45a-4edb-9ff3-61a418e99a23 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:42.723975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e544f-b45a-4edb-9ff3-61a418e99a23 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:42.724391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e544f-b45a-4edb-9ff3-61a418e99a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:42.754912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bd2e544f-b45a-4edb-9ff3-61a418e99a23 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:42.756132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bd2e544f-b45a-4edb-9ff3-61a418e99a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:42.852947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1244/2468] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:35:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:42.865607 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:42.867108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:42.977877 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa82659-3b9f-463f-9528-a2696a4c9588 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:42.982939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:42.983583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:42.983888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:42.984148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:42.994789 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:42.996429 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:43.005560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1225/2469] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:35:43.569348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f565-eb8d-44dd-8c85-5871cccbf8e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:43.569650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f565-eb8d-44dd-8c85-5871cccbf8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:43.569933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f565-eb8d-44dd-8c85-5871cccbf8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:43.603518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aa66f565-eb8d-44dd-8c85-5871cccbf8e4 None None] MFA Rules not processed for user `493d126879f44b26951920235e7f7730`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:43.606144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa66f565-eb8d-44dd-8c85-5871cccbf8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:43.651381 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1245/2470] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:35:43.723611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:43.898266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465a6bc-5eef-4c26-9bbf-5c35b4ff464c service swift] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:43.910480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:43.911389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:43.911840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:43.912391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] PATH_INFO: `/v3/users/c7a9422dd1f348c5a60af03d0128ce59` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:43.914565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a9422dd1f348c5a60af03d0128ce59)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:43.916526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:43.980287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:44.017486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a9422dd1f348c5a60af03d0128ce59'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.029295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a9422dd1f348c5a60af03d0128ce59'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.058051 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a9422dd1f348c5a60af03d0128ce59'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.061168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a9422dd1f348c5a60af03d0128ce59'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.081226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a9422dd1f348c5a60af03d0128ce59'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.104735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1226/2471] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:43 2022] DELETE /identity/v3/users/c7a9422dd1f348c5a60af03d0128ce59 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:44.106634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:44.107190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:44.107190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:44.212457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:44.214579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:44.307680 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1246/2472] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:35:44.318133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:44.319807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:44.410066 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d126-61b6-4d48-99b9-4f808e1e551e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:44.414068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90e4ed-40ce-428b-bd6f-9f44e75ae362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:44.414720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90e4ed-40ce-428b-bd6f-9f44e75ae362 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:44.415007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90e4ed-40ce-428b-bd6f-9f44e75ae362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:44.415294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90e4ed-40ce-428b-bd6f-9f44e75ae362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:44.425828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90e4ed-40ce-428b-bd6f-9f44e75ae362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:44.427291 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90e4ed-40ce-428b-bd6f-9f44e75ae362 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:44.437405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1227/2473] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:44.839329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:44.841088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:44.855667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00dab8-d4ec-4459-97bb-1cee5ddd53e5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:44.861606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:44.862760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:44.863155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:44.863569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] PATH_INFO: `/v3/projects/96fa88f82fe64faaaa774e8c9c963609` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:44.865448 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96fa88f82fe64faaaa774e8c9c963609)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:44.871072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:44.941830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:44.955526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96fa88f82fe64faaaa774e8c9c963609', 'request_id': 'req-8bc8b685-9dee-4b38-960c-58fe932f74b0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.967576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96fa88f82fe64faaaa774e8c9c963609', 'request_id': 'req-8bc8b685-9dee-4b38-960c-58fe932f74b0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:44.993542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96fa88f82fe64faaaa774e8c9c963609', 'request_id': 'req-8bc8b685-9dee-4b38-960c-58fe932f74b0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.019087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] The token cache is being invalidate because project 96fa88f82fe64faaaa774e8c9c963609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:45.019671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.020734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8bc8b685-9dee-4b38-960c-58fe932f74b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:45.032592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1247/2474] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:44 2022] DELETE /identity/v3/projects/96fa88f82fe64faaaa774e8c9c963609 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:45.033183 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.033700 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.033781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.109614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:45.112385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:45.216442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1228/2475] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:45.218593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.219707 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.219919 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.246115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:45.247481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:45.262530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.268515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1248/2476] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:45.274265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8802c6-4176-4258-8279-227f0853cd21 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.287387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.288221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.288520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.288802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.289649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:35:45.289966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.291586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.320979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1229/2477] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:35:45.322514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.333625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56d62-6dde-4da9-b98f-bd1f2a0ea92b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.338927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.339871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.340269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.340639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.341516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.343804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.389664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1249/2478] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:45.395829 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.428703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbd371-216b-4701-bf42-ecc3255379d2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.453246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:45.559520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.560556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.561052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.561763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.563629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.575229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.626179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1230/2479] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:45.630130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.643512 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8f6d3-bf1c-4078-9a2a-827bdbbf9974 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.649979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.651213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.651744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.652323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.653378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.655626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.670043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1250/2480] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:45.675534 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.695150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a06f-02ed-4192-8531-5921d0c68e60 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.704599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2689c-45d4-4dfe-b4a5-f7cb8c6c638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.705333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2689c-45d4-4dfe-b4a5-f7cb8c6c638c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.705770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2689c-45d4-4dfe-b4a5-f7cb8c6c638c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.706147 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2689c-45d4-4dfe-b4a5-f7cb8c6c638c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.706898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2689c-45d4-4dfe-b4a5-f7cb8c6c638c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.708738 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2689c-45d4-4dfe-b4a5-f7cb8c6c638c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.720356 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1231/2481] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:45.724784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.735259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b578d8c-7286-49ad-a2c0-7066c4abcf2d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.739793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.740657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.740964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.741237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] PATH_INFO: `/v3/projects/08fcec3219064371acaa7737b4e97a2b/users/e1924fd7733f485e973dc5db1896b330/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.755520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] RBAC: Authorizing `identity:create_grant(project_id=08fcec3219064371acaa7737b4e97a2b, user_id=e1924fd7733f485e973dc5db1896b330, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.758183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:45.771068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1251/2482] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:45 2022] PUT /identity/v3/projects/08fcec3219064371acaa7737b4e97a2b/users/e1924fd7733f485e973dc5db1896b330/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:45.781198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.782061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.782333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.809796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] MFA Rules not processed for user `e1924fd7733f485e973dc5db1896b330`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:45.810998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:45.876351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1232/2483] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:45.880759 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:45.953255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d552b6a-763a-49d0-a806-2460441d269b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:45.961255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:45.961994 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:45.962286 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:45.962592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:45.963296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:45.965697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.005015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1252/2484] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:45 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:46.006532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.026745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef8ac5-47b8-41f0-b7ec-82644771bfb0 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.105540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.106968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.107426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.107709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.108412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.111070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.142863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.148965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1233/2485] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:46 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:35:46.156160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a063f-7c64-4667-bc29-4c019d5a038e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.161539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.162478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.162770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.163053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.163797 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.165465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.183839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.184456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1253/2486] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:46.212533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69622d0-8c6e-4d6b-a834-e9a41dd2981e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.221162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.222735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.223247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.223976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] PATH_INFO: `/v3/projects/d497da86a255427686e9a4de8e650f7c/users/b005274632cc4c8b9aafc050280cf970/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.256522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:46.279830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d497da86a255427686e9a4de8e650f7c, user_id=b005274632cc4c8b9aafc050280cf970, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.283473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.320988 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1234/2487] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:46 2022] PUT /identity/v3/projects/d497da86a255427686e9a4de8e650f7c/users/b005274632cc4c8b9aafc050280cf970/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:35:46.325554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.336678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae00fdc-1133-49c3-849c-033a84942c69 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.388925 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:46.418177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e456a3-d7d6-424d-a37f-cf60c203dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.418767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e456a3-d7d6-424d-a37f-cf60c203dc6d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.419051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e456a3-d7d6-424d-a37f-cf60c203dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.419327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e456a3-d7d6-424d-a37f-cf60c203dc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.420325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e456a3-d7d6-424d-a37f-cf60c203dc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.423389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e456a3-d7d6-424d-a37f-cf60c203dc6d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.435090 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1254/2488] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:46.439323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.455152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d312-b6c5-4b35-b139-7f9515bb4a33 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.463118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.464408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.464798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.465226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] PATH_INFO: `/v3/projects/d497da86a255427686e9a4de8e650f7c/users/b005274632cc4c8b9aafc050280cf970/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.468384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d497da86a255427686e9a4de8e650f7c, user_id=b005274632cc4c8b9aafc050280cf970, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.472979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.508160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1235/2489] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:46 2022] PUT /identity/v3/projects/d497da86a255427686e9a4de8e650f7c/users/b005274632cc4c8b9aafc050280cf970/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:46.508813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.509241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.509721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.550316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] MFA Rules not processed for user `b005274632cc4c8b9aafc050280cf970`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:46.551715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:46.638850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1255/2490] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:46.653703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.656251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:46.753372 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761e53c-6d13-4ea2-a4dc-6172fd40f8ee admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.763552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.765304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.767153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.767519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.779098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.781127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.797243 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1236/2491] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:35:46.885625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:46.896483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52123d-f989-49db-86ef-cb8c9b898ac1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:46.901570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fef62-8098-4040-bc62-051a47e5ce05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:46.902299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fef62-8098-4040-bc62-051a47e5ce05 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:46.902595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fef62-8098-4040-bc62-051a47e5ce05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:46.902876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fef62-8098-4040-bc62-051a47e5ce05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:46.984442 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fef62-8098-4040-bc62-051a47e5ce05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:46.985953 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fef62-8098-4040-bc62-051a47e5ce05 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:46.997322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1256/2492] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:56.790924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:56.992134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=73 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:57.075399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.139038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245b321-4324-4781-9984-79363eb78494 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.143750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.144620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.144927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.145207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] PATH_INFO: `/v3/users/e1924fd7733f485e973dc5db1896b330` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.146389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1924fd7733f485e973dc5db1896b330)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.148303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.194810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1924fd7733f485e973dc5db1896b330'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.201705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1924fd7733f485e973dc5db1896b330'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.215952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1924fd7733f485e973dc5db1896b330'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.216707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1924fd7733f485e973dc5db1896b330'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.229402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1924fd7733f485e973dc5db1896b330'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.236574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1237/2493] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:57 2022] DELETE /identity/v3/users/e1924fd7733f485e973dc5db1896b330 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:57.241198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.241492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.241772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.259096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:57.260323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:57.321924 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1257/2494] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:35:57.332413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.333847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:57.393801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9e46f-9626-4688-9527-3cbddf07d5cd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.397442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.398125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.398415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.398698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.409222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.410682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.420937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1238/2495] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:57.878403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:57.880344 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:57.899787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1233912-b5a0-4315-a82b-ff984ad85c42 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:57.904801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:57.905709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:57.906005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:57.907040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] PATH_INFO: `/v3/projects/08fcec3219064371acaa7737b4e97a2b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:57.908350 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08fcec3219064371acaa7737b4e97a2b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:57.910304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:57.983105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08fcec3219064371acaa7737b4e97a2b', 'request_id': 'req-0792981a-9cf7-4bd7-abc9-cf6a4910046c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.990416 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08fcec3219064371acaa7737b4e97a2b', 'request_id': 'req-0792981a-9cf7-4bd7-abc9-cf6a4910046c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:57.998740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08fcec3219064371acaa7737b4e97a2b', 'request_id': 'req-0792981a-9cf7-4bd7-abc9-cf6a4910046c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.010551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] The token cache is being invalidate because project 08fcec3219064371acaa7737b4e97a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:58.010843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.011570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.013220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1258/2496] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:57 2022] DELETE /identity/v3/projects/08fcec3219064371acaa7737b4e97a2b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:35:58.018347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:58.039261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfd6e0-474f-4984-9405-ddb7f66a8486 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:58.061308 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:58.135901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:58.136964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.137324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.138176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] PATH_INFO: `/v3/users/b005274632cc4c8b9aafc050280cf970` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:58.139551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b005274632cc4c8b9aafc050280cf970)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:58.142627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:58.208993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b005274632cc4c8b9aafc050280cf970'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.221359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b005274632cc4c8b9aafc050280cf970'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.235499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b005274632cc4c8b9aafc050280cf970'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.236339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b005274632cc4c8b9aafc050280cf970'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.244168 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a69a0e33-1216-4cc9-b0f8-e8ce9e1d447a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b005274632cc4c8b9aafc050280cf970'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.250992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1239/2497] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:35:58 2022] DELETE /identity/v3/users/b005274632cc4c8b9aafc050280cf970 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:58.644633 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:58.646437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:58.660625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792981a-9cf7-4bd7-abc9-cf6a4910046c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:58.780534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:58.783303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.783704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.784514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] PATH_INFO: `/v3/projects/d497da86a255427686e9a4de8e650f7c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:58.787045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d497da86a255427686e9a4de8e650f7c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:58.791529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:58.846781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:58.900472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd497da86a255427686e9a4de8e650f7c', 'request_id': 'req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.925490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd497da86a255427686e9a4de8e650f7c', 'request_id': 'req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.941786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd497da86a255427686e9a4de8e650f7c', 'request_id': 'req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.955186 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] The token cache is being invalidate because project d497da86a255427686e9a4de8e650f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:35:58.955558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.956162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a164f7f-dc3b-40a3-97e8-c283f6c586ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:35:58.957804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1259/2498] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:35:58 2022] DELETE /identity/v3/projects/d497da86a255427686e9a4de8e650f7c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:58.963076 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:58.963376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:58.963687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.005379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:59.006638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:59.108204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1240/2499] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:59.111123 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.111880 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.112347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.135117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:59.136666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:59.149230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1260/2500] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:59.151234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.163698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a744cbc-bead-41c5-a8d1-07940d138caf None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.173229 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.174005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.174302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.174583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.175328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:35:59.175642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.177394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.190819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1241/2501] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:35:59.194962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.204906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3954d-fe78-4d73-9c3b-408935cfd76d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.209945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.210322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.210606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.210884 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.211595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.213581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.245513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1261/2502] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:35:59.249225 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.260184 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c4ab5-58ce-43a4-b2b0-f85ef43efc2a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.289481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:35:59.328666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.329509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.329809 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.330827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.331554 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.335695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.358497 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1242/2503] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:35:59.363222 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.374408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a2344-6d8a-41d3-acb2-5cdced90ed57 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.379846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.380611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.380991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.381334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.382155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.383843 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.395649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1262/2504] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:59.399748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.410826 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54161c5-7152-4c68-9548-a81ca91db999 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.415810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a58b9c-6eac-4cc1-8f0f-db981ba418b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.416486 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a58b9c-6eac-4cc1-8f0f-db981ba418b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.416773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a58b9c-6eac-4cc1-8f0f-db981ba418b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.417084 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a58b9c-6eac-4cc1-8f0f-db981ba418b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.417822 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a58b9c-6eac-4cc1-8f0f-db981ba418b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.419442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a58b9c-6eac-4cc1-8f0f-db981ba418b8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.431994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1243/2505] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:59.435935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.446845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18566d7-843d-4024-8a59-b79fa8021f6f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.451752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.452667 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.453519 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.453767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] PATH_INFO: `/v3/projects/2b8a71b21c4d4a939e63f68ef8e16c63/users/183dde7ae7784d95ae14bde53d8e6cc6/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.471354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8a71b21c4d4a939e63f68ef8e16c63, user_id=183dde7ae7784d95ae14bde53d8e6cc6, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.474270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.488850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1263/2506] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:59 2022] PUT /identity/v3/projects/2b8a71b21c4d4a939e63f68ef8e16c63/users/183dde7ae7784d95ae14bde53d8e6cc6/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:59.499589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.499894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.500174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.529792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] MFA Rules not processed for user `183dde7ae7784d95ae14bde53d8e6cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:35:59.531044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:35:59.599851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1244/2507] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:35:59.604243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.671812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee3fd2-0e15-4293-b90f-b9b849e585e8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.679435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.680135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.680424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.680706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.681460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.683451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.722832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1264/2508] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:35:59.726546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.740286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8b425-5222-4b61-8fff-91f1d437458b None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.809997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.810962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.811234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.811585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.813060 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.815073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.839694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1245/2509] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:35:59 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:35:59.843910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.855388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986747d-654a-449d-9a70-b84faa506806 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.864688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.865631 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.866095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.866675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.867613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.871719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.886248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1265/2510] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:35:59.891218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.907241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99b26e-e055-4da1-bfde-6dfd6a537b05 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:35:59.911562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:35:59.912981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:35:59.913265 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:35:59.913797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] PATH_INFO: `/v3/projects/f85ad135bbb54a5bb99219ffee3285f1/users/f0cdcaeb124b4c418b6f5eb70d5e577a/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:35:59.936231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85ad135bbb54a5bb99219ffee3285f1, user_id=f0cdcaeb124b4c418b6f5eb70d5e577a, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:35:59.941170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:35:59.959618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1246/2511] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:35:59 2022] PUT /identity/v3/projects/f85ad135bbb54a5bb99219ffee3285f1/users/f0cdcaeb124b4c418b6f5eb70d5e577a/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:35:59.963338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:35:59.985415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396aa39a-0ee8-4174-94d0-4d780351ce1e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.018393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:00.085932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042bfdb-ab16-4f1c-9a63-d287006be90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.086638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042bfdb-ab16-4f1c-9a63-d287006be90e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.087185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042bfdb-ab16-4f1c-9a63-d287006be90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.087503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042bfdb-ab16-4f1c-9a63-d287006be90e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.088332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042bfdb-ab16-4f1c-9a63-d287006be90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.089996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042bfdb-ab16-4f1c-9a63-d287006be90e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.105317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1266/2512] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:00.106234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.114469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:00.129616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f24f24-8a84-483b-9f1f-cf8f8bc6d2f0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.134909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.136021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.136455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.136911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] PATH_INFO: `/v3/projects/f85ad135bbb54a5bb99219ffee3285f1/users/f0cdcaeb124b4c418b6f5eb70d5e577a/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.139303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85ad135bbb54a5bb99219ffee3285f1, user_id=f0cdcaeb124b4c418b6f5eb70d5e577a, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.143289 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.158102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1247/2513] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:00 2022] PUT /identity/v3/projects/f85ad135bbb54a5bb99219ffee3285f1/users/f0cdcaeb124b4c418b6f5eb70d5e577a/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:00.169619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.170753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.171060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.231838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] MFA Rules not processed for user `f0cdcaeb124b4c418b6f5eb70d5e577a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:00.233116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:00.331167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1267/2514] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:00.343000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.346657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:00.444914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf7de9-abb9-4382-93a8-54757c699e21 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.454413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ecf16-8a5e-4e7d-95d3-53200ed486b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.455216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ecf16-8a5e-4e7d-95d3-53200ed486b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.455533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ecf16-8a5e-4e7d-95d3-53200ed486b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.455813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ecf16-8a5e-4e7d-95d3-53200ed486b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.467078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ecf16-8a5e-4e7d-95d3-53200ed486b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.468584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ecf16-8a5e-4e7d-95d3-53200ed486b5 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.480190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1248/2515] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:00.554120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:00.565239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05f2f5-64ea-4ad7-9cd4-7a3a214777b4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:00.569128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:00.569844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.570199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.570545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.634575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:00.636177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:00.650331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1268/2516] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:00 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:00.924763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:00.925090 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:00.925379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:00.964364 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:00.965670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:01.032470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1249/2517] 10.222.0.79 () {58 vars in 967 bytes} [Wed Jul 20 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:01.047083 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:01.048609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:01.130456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:01.182094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9302583-64f8-45d2-ac74-fa9aadd71417 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:01.191743 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:01.192242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:01.192529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:01.192807 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:01.208635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:01.210246 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:01.219691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1269/2518] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:36:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:36:09.274705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:09.276102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:09.358517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403d771-9b45-49e7-a75a-14e579322159 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:09.362517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:09.363166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:09.363569 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:09.363851 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:09.365666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:09.386995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:09.431667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:09.433065 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835deca-406c-4a92-b460-aaa70b865b69 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:09.443078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1250/2519] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:11.039168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:11.050278 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad438648-9b69-4aee-a5d3-fc8340d984ea service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:11.054997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:11.055873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:11.056153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:11.056433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] PATH_INFO: `/v3/users/183dde7ae7784d95ae14bde53d8e6cc6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:11.057829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=183dde7ae7784d95ae14bde53d8e6cc6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:11.059772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:11.124452 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '183dde7ae7784d95ae14bde53d8e6cc6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.134265 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '183dde7ae7784d95ae14bde53d8e6cc6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.149979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '183dde7ae7784d95ae14bde53d8e6cc6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.150542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '183dde7ae7784d95ae14bde53d8e6cc6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.161716 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '183dde7ae7784d95ae14bde53d8e6cc6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.174588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1270/2520] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:11 2022] DELETE /identity/v3/users/183dde7ae7784d95ae14bde53d8e6cc6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:11.176057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:11.176560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:11.176686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:11.196588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:11.198006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:11.267567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1251/2521] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:11.278037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:11.279600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:11.350065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f14d99-9c64-4624-8ba2-14cfb5d6e92b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:11.353789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:11.354459 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:11.354748 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:11.355025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:11.366404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:11.367855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:11.379335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1271/2522] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:11.699933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:11.701413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:11.719348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f12a8-4743-44fd-9986-c7d6545cd8a8 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:11.725844 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:11.726485 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:11.726722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:11.726998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] PATH_INFO: `/v3/projects/2b8a71b21c4d4a939e63f68ef8e16c63` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:11.728176 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8a71b21c4d4a939e63f68ef8e16c63)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:11.730140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:11.776072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8a71b21c4d4a939e63f68ef8e16c63', 'request_id': 'req-0c9de316-05d3-4849-8659-60d03f609528'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.783200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8a71b21c4d4a939e63f68ef8e16c63', 'request_id': 'req-0c9de316-05d3-4849-8659-60d03f609528'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.790886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8a71b21c4d4a939e63f68ef8e16c63', 'request_id': 'req-0c9de316-05d3-4849-8659-60d03f609528'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.804762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] The token cache is being invalidate because project 2b8a71b21c4d4a939e63f68ef8e16c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:11.805067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.805760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.807556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1252/2523] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:11 2022] DELETE /identity/v3/projects/2b8a71b21c4d4a939e63f68ef8e16c63 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:11.811708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:11.822061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367bf0-5512-4662-bfde-6e2f10741b86 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:11.872262 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:11.887023 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:11.888243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:11.888665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:11.889400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] PATH_INFO: `/v3/users/f0cdcaeb124b4c418b6f5eb70d5e577a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:11.891805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cdcaeb124b4c418b6f5eb70d5e577a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:11.900986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:11.981024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cdcaeb124b4c418b6f5eb70d5e577a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:11.990148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cdcaeb124b4c418b6f5eb70d5e577a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.009269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cdcaeb124b4c418b6f5eb70d5e577a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.010032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cdcaeb124b4c418b6f5eb70d5e577a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.018655 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e0097a-f79d-43b1-af58-53355e9195d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cdcaeb124b4c418b6f5eb70d5e577a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.028745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1272/2524] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:11 2022] DELETE /identity/v3/users/f0cdcaeb124b4c418b6f5eb70d5e577a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:12.347292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:12.350305 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:12.390691 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9de316-05d3-4849-8659-60d03f609528 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:12.440860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:12.461004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:12.462043 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:12.462432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:12.462818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] PATH_INFO: `/v3/projects/f85ad135bbb54a5bb99219ffee3285f1` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:12.464139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85ad135bbb54a5bb99219ffee3285f1)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:12.466304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:12.559911 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85ad135bbb54a5bb99219ffee3285f1', 'request_id': 'req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.568985 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85ad135bbb54a5bb99219ffee3285f1', 'request_id': 'req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.578886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85ad135bbb54a5bb99219ffee3285f1', 'request_id': 'req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.595466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] The token cache is being invalidate because project f85ad135bbb54a5bb99219ffee3285f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:12.595764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.596447 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-a2cf4e9a-46fa-4c79-bd8e-0665dddbee10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:12.598273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1253/2525] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:12 2022] DELETE /identity/v3/projects/f85ad135bbb54a5bb99219ffee3285f1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:12.603151 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:12.603708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:12.603708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:12.630999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:12.632572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:12.728370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1273/2526] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:12.738502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:12.738943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:12.739347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:12.778038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:12.779392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:12.794241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1254/2527] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:12.799102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:12.812250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c4412-eb20-437d-929c-c25b171f2134 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:12.818021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:12.818993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:12.819455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:12.819975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:12.820850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:12.825495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:12.827543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:12.843049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1274/2528] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:12.851253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:12.863109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175f9a5-07b7-4d23-a79d-1c852f1ae343 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:12.868274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:12.869037 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:12.869326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:12.869964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:12.870310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:12.872261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:12.904099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1255/2529] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:12 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:12.915528 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:12.928917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eb92d-ba31-4d4e-9319-1e5916005ea3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:12.991822 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:13.001432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.002267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.002865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.003686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.004626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.007250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.041112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1275/2530] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:12 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:13.042433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.053746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c2aa4-5c5c-4469-bd73-d3d129c73454 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.059009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.059742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.060094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.060436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.061270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.063637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.077520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1256/2531] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:13.079783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.093197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3881b-3bce-4596-8021-db927c345d68 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.099574 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f94d9-ae17-4e37-85f7-01dbf6f06142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.100510 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f94d9-ae17-4e37-85f7-01dbf6f06142 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.100791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f94d9-ae17-4e37-85f7-01dbf6f06142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.101296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f94d9-ae17-4e37-85f7-01dbf6f06142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.101876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f94d9-ae17-4e37-85f7-01dbf6f06142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.103615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f94d9-ae17-4e37-85f7-01dbf6f06142 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.115216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1276/2532] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:13.119594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.132137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fa01f-5fbb-451b-8a82-600b04c0f2d6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.138327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.139240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.139533 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.139840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] PATH_INFO: `/v3/projects/3342d41b891d49758fc3793f5ec3a15e/users/a7aae0704b02401ea5d9c39eece6bd36/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.160276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3342d41b891d49758fc3793f5ec3a15e, user_id=a7aae0704b02401ea5d9c39eece6bd36, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.163439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.184230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1257/2533] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:13 2022] PUT /identity/v3/projects/3342d41b891d49758fc3793f5ec3a15e/users/a7aae0704b02401ea5d9c39eece6bd36/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:13.196258 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.196551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.196829 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.229238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] MFA Rules not processed for user `a7aae0704b02401ea5d9c39eece6bd36`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:13.230537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.315343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1277/2534] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:13.319959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.353460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.355173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.387087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb02e-a353-4eff-9cad-6fc47b93568d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.395480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.396297 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.396716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.397095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.397798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.400679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.416992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:13.436930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1258/2535] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:13 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:13.436930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.436930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.477078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0bdd0-7786-498d-b138-12377d8b8c6f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.505403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0674c1-8ed5-4060-9cad-283ee328407b admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.510825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.511819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.512253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.512545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] PATH_INFO: `/v3/users/3f7758d298f642138b71aa422d0344da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.514015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7758d298f642138b71aa422d0344da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.517406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.529027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:13.551391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.552063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.552351 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.552657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.554649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.566558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.568200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.569319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7758d298f642138b71aa422d0344da'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.576120 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7758d298f642138b71aa422d0344da'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.578389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1278/2536] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:36:13.579343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.588812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7758d298f642138b71aa422d0344da'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.589679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7758d298f642138b71aa422d0344da'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.590593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85026d3f-2942-4acd-82b2-ef17eb2860c6 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.596813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.597589 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.597902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.598249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.598945 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.600852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.602770 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7758d298f642138b71aa422d0344da'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.619742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1259/2537] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:13 2022] DELETE /identity/v3/users/3f7758d298f642138b71aa422d0344da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:13.627665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1279/2538] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:13 2022] POST /identity/v3/users => generated 299 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:13.631362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.633526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.720893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc5ec7-23c5-437f-ac89-5b73a27ef65d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.725817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.726879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.726879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.727142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.727840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.729470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.739888 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1260/2539] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:36:13.744105 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.754458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0a2b-67ea-44e6-9433-b5d42676317e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.759143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.760037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.760315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.760590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] PATH_INFO: `/v3/projects/b0cedd0706a44e2294309a5137525446/users/e886540eda314a57a3836b428b41b058/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.777615 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0cedd0706a44e2294309a5137525446, user_id=e886540eda314a57a3836b428b41b058, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.780291 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.795542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1280/2540] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:13 2022] PUT /identity/v3/projects/b0cedd0706a44e2294309a5137525446/users/e886540eda314a57a3836b428b41b058/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:13.799706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.809860 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa23160-cc49-48f9-a3e2-c69ff4230482 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.814737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:13.826335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.828782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.879579 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.880256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.880546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.880916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.881716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.883334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.897817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1261/2541] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:36:13.898867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:13.899315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ad3c-316b-4321-a8bf-efafd4695ce3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.904263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.905158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.905445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.905725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] PATH_INFO: `/v3/projects/02f994a0231b4176b8181d13d58ee9ec` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.907679 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f994a0231b4176b8181d13d58ee9ec)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.909606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0446d0d-8f89-46fb-b981-494945a27c61 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:13.909724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.914748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ee5d2-bb6a-41ee-9271-5c77d1ec3c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:13.915630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee5d2-bb6a-41ee-9271-5c77d1ec3c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.915919 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee5d2-bb6a-41ee-9271-5c77d1ec3c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.916195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee5d2-bb6a-41ee-9271-5c77d1ec3c32 admin admin] PATH_INFO: `/v3/projects/b0cedd0706a44e2294309a5137525446/users/e886540eda314a57a3836b428b41b058/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.918441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ee5d2-bb6a-41ee-9271-5c77d1ec3c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0cedd0706a44e2294309a5137525446, user_id=e886540eda314a57a3836b428b41b058, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:13.921128 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ee5d2-bb6a-41ee-9271-5c77d1ec3c32 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:13.934755 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1262/2542] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:13 2022] PUT /identity/v3/projects/b0cedd0706a44e2294309a5137525446/users/e886540eda314a57a3836b428b41b058/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:13.946292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:13.946591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:13.946886 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:13.971245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f994a0231b4176b8181d13d58ee9ec', 'request_id': 'req-ba872d00-1f53-4790-af8a-a47b368fa430'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.977951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] MFA Rules not processed for user `e886540eda314a57a3836b428b41b058`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:13.978124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f994a0231b4176b8181d13d58ee9ec', 'request_id': 'req-ba872d00-1f53-4790-af8a-a47b368fa430'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:13.979587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:13.989158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f994a0231b4176b8181d13d58ee9ec', 'request_id': 'req-ba872d00-1f53-4790-af8a-a47b368fa430'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:14.005130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] The token cache is being invalidate because project 02f994a0231b4176b8181d13d58ee9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:14.005666 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:14.006711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba872d00-1f53-4790-af8a-a47b368fa430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:14.009053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1281/2543] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:13 2022] DELETE /identity/v3/projects/02f994a0231b4176b8181d13d58ee9ec => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:14.014371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef65020-40ce-43be-a6f3-31223be90cc3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.014957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef65020-40ce-43be-a6f3-31223be90cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.016166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef65020-40ce-43be-a6f3-31223be90cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.038599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2ef65020-40ce-43be-a6f3-31223be90cc3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:14.039798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ef65020-40ce-43be-a6f3-31223be90cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.063760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:14.072253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1263/2544] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:14.085395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.087380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.132423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1282/2545] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:14.139608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.140231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.140537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.162731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:14.163966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.173112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1283/2546] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:14.177429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.188054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14278c01-9aa7-411f-b25a-e36e9ab66357 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.189661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fcb87-4725-47f4-8cce-e5b1c72e6242 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.192391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.193095 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.193386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.193670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.197020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.197020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.197020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.197020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.197462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:14.197462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.198598 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.204698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.206200 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.211894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:14.212026 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1284/2547] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:36:14.216701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.221979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1264/2548] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:14.229091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa8289-dd2d-4ceb-bb73-031120801ece admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.235359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.236032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.236328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.236606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.237976 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.240657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.281927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1285/2549] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:14 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:14.286464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.300563 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e88f0-e11c-4eea-8df4-9c04a716870b service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.328039 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.332379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.371825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.372510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.372792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.373226 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.373928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.376350 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.399772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1265/2550] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:14 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:14.404736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.409122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f275d1-13e8-47d2-a748-6e0bbfb1548a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.416217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.416891 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.417263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.417385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ffa65-6a97-4110-a030-380e1334489c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.417614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.419364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.422787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.423455 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.423745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.424110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.424849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.426526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.440202 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1266/2551] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:36:14.441564 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.455069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6bd19-4abf-41dd-8984-99432fa73583 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.460612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.462102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.462391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.462669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.463520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.465110 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.485282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1267/2552] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:36:14.489442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.496729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.498209 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547d22d-69ed-42da-b20c-7ce00a713338 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.505769 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121103-a50d-4408-aef2-4e67b080ae64 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.510091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1286/2553] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:36:14.512397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.513420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.513706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.513982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] PATH_INFO: `/v3/projects/a1bfa2d4b4084aa294fbd546ebc0f8c5/users/90dedb6ffa9f40fe905f60b023aeb92e/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.534559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bfa2d4b4084aa294fbd546ebc0f8c5, user_id=90dedb6ffa9f40fe905f60b023aeb92e, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.537774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.554526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1268/2554] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:14 2022] PUT /identity/v3/projects/a1bfa2d4b4084aa294fbd546ebc0f8c5/users/90dedb6ffa9f40fe905f60b023aeb92e/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:14.559883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.560234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.560472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.588537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:14.591629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.622613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:14.658629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1287/2555] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:14.670405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.750650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdb262-7f09-40ff-b264-895becfa65bf admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.754925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8007d37-8ab0-43b4-8638-c45a208c507b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.755581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8007d37-8ab0-43b4-8638-c45a208c507b service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.755870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8007d37-8ab0-43b4-8638-c45a208c507b service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.756315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8007d37-8ab0-43b4-8638-c45a208c507b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.766752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8007d37-8ab0-43b4-8638-c45a208c507b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.768202 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8007d37-8ab0-43b4-8638-c45a208c507b service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:14.779508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1269/2556] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:14.909064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:14.911314 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:14.923702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e3f1f-9f06-4ad2-8a03-f3c64c52d4fe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:14.929229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:14.930067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:14.930364 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:14.930856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] PATH_INFO: `/v3/users/a7aae0704b02401ea5d9c39eece6bd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:14.932171 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7aae0704b02401ea5d9c39eece6bd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:14.934662 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:15.002241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7aae0704b02401ea5d9c39eece6bd36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:15.012774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7aae0704b02401ea5d9c39eece6bd36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:15.030766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7aae0704b02401ea5d9c39eece6bd36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:15.032241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7aae0704b02401ea5d9c39eece6bd36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:15.047483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7aae0704b02401ea5d9c39eece6bd36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:15.056446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1288/2557] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:14 2022] DELETE /identity/v3/users/a7aae0704b02401ea5d9c39eece6bd36 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:15.060745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:15.061199 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:15.061583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:15.082080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:15.083753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:15.146899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:15.170935 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1270/2558] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:15.185016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:15.187813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:15.270495 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b652e-55cc-4798-b462-e8af102b8fee admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:15.277774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:15.279871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:15.280199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:15.280450 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:15.294827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:15.297394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:15.308772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1289/2559] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:36:16.087715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:16.090201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:16.104207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f521d-5f3e-4fcb-9992-e129ef443109 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:16.124530 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:16.126314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:16.126599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:16.126880 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] PATH_INFO: `/v3/projects/3342d41b891d49758fc3793f5ec3a15e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:16.128428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3342d41b891d49758fc3793f5ec3a15e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:16.130428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:16.199426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3342d41b891d49758fc3793f5ec3a15e', 'request_id': 'req-c684a465-952c-4d86-96b3-b379a8728301'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.216539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3342d41b891d49758fc3793f5ec3a15e', 'request_id': 'req-c684a465-952c-4d86-96b3-b379a8728301'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.228386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3342d41b891d49758fc3793f5ec3a15e', 'request_id': 'req-c684a465-952c-4d86-96b3-b379a8728301'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.240792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] The token cache is being invalidate because project 3342d41b891d49758fc3793f5ec3a15e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:16.241551 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.242328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.244307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1271/2560] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:16 2022] DELETE /identity/v3/projects/3342d41b891d49758fc3793f5ec3a15e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:16.249441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:16.267081 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e925d-6982-47d2-a58b-9a508c03c2fa service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:16.333587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:16.355849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:16.357684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:16.358334 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:16.358527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] PATH_INFO: `/v3/users/e886540eda314a57a3836b428b41b058` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:16.361337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e886540eda314a57a3836b428b41b058)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:16.363795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:16.427514 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e886540eda314a57a3836b428b41b058'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.435968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e886540eda314a57a3836b428b41b058'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.449833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e886540eda314a57a3836b428b41b058'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.450651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e886540eda314a57a3836b428b41b058'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.459962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-210d9776-4ca3-49d8-b411-2b2219aa3ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e886540eda314a57a3836b428b41b058'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:16.467340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1290/2561] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:16 2022] DELETE /identity/v3/users/e886540eda314a57a3836b428b41b058 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:16.853916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:16.855900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:16.872438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684a465-952c-4d86-96b3-b379a8728301 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:16.944802 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:16.945667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:16.946005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:16.946296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] PATH_INFO: `/v3/projects/b0cedd0706a44e2294309a5137525446` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:16.947493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0cedd0706a44e2294309a5137525446)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:16.949507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.015841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0cedd0706a44e2294309a5137525446', 'request_id': 'req-49f0be0a-14d7-4054-8e6e-c46999465c30'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.022933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0cedd0706a44e2294309a5137525446', 'request_id': 'req-49f0be0a-14d7-4054-8e6e-c46999465c30'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.034086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0cedd0706a44e2294309a5137525446', 'request_id': 'req-49f0be0a-14d7-4054-8e6e-c46999465c30'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.050610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:17.056907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] The token cache is being invalidate because project b0cedd0706a44e2294309a5137525446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:17.057217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.058638 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-49f0be0a-14d7-4054-8e6e-c46999465c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:17.067678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1272/2562] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:16 2022] DELETE /identity/v3/projects/b0cedd0706a44e2294309a5137525446 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:17.069499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.069795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.070077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.093799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:17.096657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:17.187693 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1291/2563] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:17.196036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.196581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.197402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.224401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:17.226711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:17.241748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1273/2564] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:17.243068 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.255322 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79b87e-9fb5-4ba4-8be0-8207b335160d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.261005 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.261433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.261823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.262204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.263064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:17.263481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.265775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.289019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1292/2565] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:17.291273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.304540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaeb330-55e6-4185-b4f5-2cd28bcd2a65 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.309799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.310597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.310984 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.311363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.312264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.314640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.359903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1274/2566] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:17 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:17.361367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.380177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde9599-cd71-4f1d-acd0-8e6334f39329 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.461996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:17.478029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.478845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.479238 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.479855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.480817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.483189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.522856 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1293/2567] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:17 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:17.527479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.544466 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4f49-0338-4f7e-b29d-9cb5e3570316 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.557008 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.558014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.558406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.558805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.560740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.567160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.591903 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1275/2568] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:17.595359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.607708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63513cea-9638-4362-aadc-9d161a93ecb9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.614730 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92b830-9846-4ad6-a24f-e3ae03939098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.616045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92b830-9846-4ad6-a24f-e3ae03939098 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.616573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92b830-9846-4ad6-a24f-e3ae03939098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.617295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92b830-9846-4ad6-a24f-e3ae03939098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.618259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92b830-9846-4ad6-a24f-e3ae03939098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.620086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92b830-9846-4ad6-a24f-e3ae03939098 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.635587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1294/2569] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:17.639669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.651193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5229a95-960f-4b70-a6a1-cc998847610a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:17.660557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:17.661532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.661819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.662103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] PATH_INFO: `/v3/projects/3e29c5d69ce6459f852b705a46a821d9/users/1ea422eb8e4f4a829d0499e10d5294c5/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.689720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e29c5d69ce6459f852b705a46a821d9, user_id=1ea422eb8e4f4a829d0499e10d5294c5, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:17.696440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:17.728201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1276/2570] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:17 2022] PUT /identity/v3/projects/3e29c5d69ce6459f852b705a46a821d9/users/1ea422eb8e4f4a829d0499e10d5294c5/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:17.740899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:17.741479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:17.742832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:17.795116 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] MFA Rules not processed for user `1ea422eb8e4f4a829d0499e10d5294c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:17.798296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:17.900774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1295/2571] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:17.901872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:17.998664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b085834-a694-49c2-8023-625be1edbeae admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.010848 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.011538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.012376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.012376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.012795 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.015351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.065021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1277/2572] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:17 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:18.069477 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.083400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650906e-9e43-4f03-bb86-d89de656fe93 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.154675 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.155509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.156317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.156594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.157561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.159611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.198442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.199201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1296/2573] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:18 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:18.216414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f504d-93ec-44e8-85c3-fc2f9e509b08 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.232152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.234432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.235178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.236155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.238269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.242026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.259858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1278/2574] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:18.264210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.276096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ff98-72a6-43fd-8c67-dd7509bc67cc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.283644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.284409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.285315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.285315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] PATH_INFO: `/v3/projects/ad713ecc372349788b7340021ae3f3df/users/ac71df536c924c5fa0ac2bfdc2ad3d67/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.303597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad713ecc372349788b7340021ae3f3df, user_id=ac71df536c924c5fa0ac2bfdc2ad3d67, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.307015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.322905 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1297/2575] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:18 2022] PUT /identity/v3/projects/ad713ecc372349788b7340021ae3f3df/users/ac71df536c924c5fa0ac2bfdc2ad3d67/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:18.329585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.341204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029bc5-ef34-4cea-8937-96e204cb9e20 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.414833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9950-1d72-4263-a290-e5b039f33973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.415528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9950-1d72-4263-a290-e5b039f33973 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.415816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9950-1d72-4263-a290-e5b039f33973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.416172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9950-1d72-4263-a290-e5b039f33973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.416945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e9950-1d72-4263-a290-e5b039f33973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.418696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e9950-1d72-4263-a290-e5b039f33973 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.435209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1279/2576] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:18.436213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.449719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b081-3291-4318-a125-da275d88ae5e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.460215 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.461316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.461651 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.461935 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] PATH_INFO: `/v3/projects/ad713ecc372349788b7340021ae3f3df/users/ac71df536c924c5fa0ac2bfdc2ad3d67/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.464230 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad713ecc372349788b7340021ae3f3df, user_id=ac71df536c924c5fa0ac2bfdc2ad3d67, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.469957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.483900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1298/2577] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:18 2022] PUT /identity/v3/projects/ad713ecc372349788b7340021ae3f3df/users/ac71df536c924c5fa0ac2bfdc2ad3d67/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:18.557491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.557786 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.558072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.573013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:18.594220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] MFA Rules not processed for user `ac71df536c924c5fa0ac2bfdc2ad3d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:18.596052 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:18.731397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1280/2578] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:18.745254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:18.747189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:18.770942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:18.831606 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc48ed-269b-493f-b532-d6762d5699ae admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:18.839387 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667844f0-c760-4b3e-8abb-bbc27c23cf91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:18.840299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667844f0-c760-4b3e-8abb-bbc27c23cf91 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:18.840592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667844f0-c760-4b3e-8abb-bbc27c23cf91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:18.840870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667844f0-c760-4b3e-8abb-bbc27c23cf91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:18.853486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667844f0-c760-4b3e-8abb-bbc27c23cf91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:18.855062 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667844f0-c760-4b3e-8abb-bbc27c23cf91 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:18.869630 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1299/2579] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:19.378894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:19.454309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9358962-7fa3-4380-a669-958e5bb830f2 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:19.459762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:19.461307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:19.461779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:19.462322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] PATH_INFO: `/v3/users/1ea422eb8e4f4a829d0499e10d5294c5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:19.463808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea422eb8e4f4a829d0499e10d5294c5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:19.465807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:19.529730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea422eb8e4f4a829d0499e10d5294c5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:19.541169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea422eb8e4f4a829d0499e10d5294c5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:19.556781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea422eb8e4f4a829d0499e10d5294c5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:19.558148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea422eb8e4f4a829d0499e10d5294c5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:19.570026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea422eb8e4f4a829d0499e10d5294c5'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:19.581098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1281/2580] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:19 2022] DELETE /identity/v3/users/1ea422eb8e4f4a829d0499e10d5294c5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:19.583143 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:19.583639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:19.583995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:19.606443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:19.608596 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:19.680501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1300/2581] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:19.693720 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:19.695559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:19.730654 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:19.797642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8892b-9fb3-44ef-b596-765d28c98afd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:19.801114 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:19.802040 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:19.802400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:19.802746 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:19.818385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:19.820752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:19.835319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1282/2582] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:20.166544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:20.168268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:20.182956 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea54617-362c-44d6-b3c7-79f0d98296ee None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:20.189871 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:20.190868 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:20.191219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:20.191568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] PATH_INFO: `/v3/projects/3e29c5d69ce6459f852b705a46a821d9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:20.192951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e29c5d69ce6459f852b705a46a821d9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:20.195268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:20.268181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e29c5d69ce6459f852b705a46a821d9', 'request_id': 'req-b5b66f90-5704-4261-9aa5-0cf922c19b2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.277078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e29c5d69ce6459f852b705a46a821d9', 'request_id': 'req-b5b66f90-5704-4261-9aa5-0cf922c19b2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.286408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e29c5d69ce6459f852b705a46a821d9', 'request_id': 'req-b5b66f90-5704-4261-9aa5-0cf922c19b2e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.308040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] The token cache is being invalidate because project 3e29c5d69ce6459f852b705a46a821d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:20.308410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.309100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b5b66f90-5704-4261-9aa5-0cf922c19b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.310745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1301/2583] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:20 2022] DELETE /identity/v3/projects/3e29c5d69ce6459f852b705a46a821d9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:20.315536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:20.325540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e705df8-30a6-4aef-8008-5158c2a309b7 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:20.402639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:20.403532 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:20.403885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:20.404169 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] PATH_INFO: `/v3/users/ac71df536c924c5fa0ac2bfdc2ad3d67` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:20.405413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac71df536c924c5fa0ac2bfdc2ad3d67)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:20.407825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:20.479599 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac71df536c924c5fa0ac2bfdc2ad3d67'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.488416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac71df536c924c5fa0ac2bfdc2ad3d67'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.504409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac71df536c924c5fa0ac2bfdc2ad3d67'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.506061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac71df536c924c5fa0ac2bfdc2ad3d67'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.521817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac71df536c924c5fa0ac2bfdc2ad3d67'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:20.529240 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1283/2584] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:20 2022] DELETE /identity/v3/users/ac71df536c924c5fa0ac2bfdc2ad3d67 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:20.624142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:20.625340 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:20.626175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:20.670588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] MFA Rules not processed for user `90dedb6ffa9f40fe905f60b023aeb92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:20.672531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:20.756176 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:20.808338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1302/2585] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:20.824068 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:20.825623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:20.876964 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:20.916959 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:20.918491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:20.961610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d92de-0eee-41a4-a2c6-92e1f6031a3c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:20.967304 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f29d8-cef5-4738-87c3-b92549fc11a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:20.968075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f29d8-cef5-4738-87c3-b92549fc11a0 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:20.969093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f29d8-cef5-4738-87c3-b92549fc11a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:20.969420 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f29d8-cef5-4738-87c3-b92549fc11a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:20.978939 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f29d8-cef5-4738-87c3-b92549fc11a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:20.982167 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f29d8-cef5-4738-87c3-b92549fc11a0 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:20.995430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1284/2586] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:21.003722 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d04f48-35cc-4c51-b2be-beb9e5831698 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.010686 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.011920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.012832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.012832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] PATH_INFO: `/v3/projects/ad713ecc372349788b7340021ae3f3df` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.014148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad713ecc372349788b7340021ae3f3df)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.016993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.081183 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad713ecc372349788b7340021ae3f3df', 'request_id': 'req-bee726d7-6489-49e5-be0c-beeedddd8c94'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:21.091267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad713ecc372349788b7340021ae3f3df', 'request_id': 'req-bee726d7-6489-49e5-be0c-beeedddd8c94'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:21.100270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad713ecc372349788b7340021ae3f3df', 'request_id': 'req-bee726d7-6489-49e5-be0c-beeedddd8c94'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:21.113361 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] The token cache is being invalidate because project ad713ecc372349788b7340021ae3f3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:21.113773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:21.114509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bee726d7-6489-49e5-be0c-beeedddd8c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:21.116756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1303/2587] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:20 2022] DELETE /identity/v3/projects/ad713ecc372349788b7340021ae3f3df => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:21.122012 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5d123-af77-4014-aa66-752891f48104 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.122519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5d123-af77-4014-aa66-752891f48104 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.122799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5d123-af77-4014-aa66-752891f48104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.148406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-68a5d123-af77-4014-aa66-752891f48104 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:21.150190 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-68a5d123-af77-4014-aa66-752891f48104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:21.240083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1285/2588] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:21.248177 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836597c-92af-44db-846c-27b50cfda066 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.248657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836597c-92af-44db-846c-27b50cfda066 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.250345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836597c-92af-44db-846c-27b50cfda066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.286201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e836597c-92af-44db-846c-27b50cfda066 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:21.287864 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e836597c-92af-44db-846c-27b50cfda066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:21.307886 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1304/2589] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:21.313318 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5d123-af77-4014-aa66-752891f48104 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.323965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5d123-af77-4014-aa66-752891f48104 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.332899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.333238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.333335 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.333595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.334674 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:21.334993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.339141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.352940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1286/2590] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:36:21.360174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836597c-92af-44db-846c-27b50cfda066 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.371909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836597c-92af-44db-846c-27b50cfda066 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.379162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.379950 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.380988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.380988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.381509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.383708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.417961 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1305/2591] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:21.422998 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.436403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb81ed-0eea-47d6-be06-f57869fd04a0 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.519332 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.520266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.520574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.521282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.522006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.523961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.548819 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1287/2592] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:21.553180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.567120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77826972-5dd2-4f0c-ab9e-034bb5398e53 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.577319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.578740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.579029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.579294 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.580072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.581926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.601613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1306/2593] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:21.606454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.619207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03449-3830-4adb-be0c-9d50d350d554 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.626454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078db0d-b459-4557-9188-8803bb3451fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.627415 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078db0d-b459-4557-9188-8803bb3451fb admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.627704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078db0d-b459-4557-9188-8803bb3451fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.627975 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078db0d-b459-4557-9188-8803bb3451fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.628729 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078db0d-b459-4557-9188-8803bb3451fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.630488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078db0d-b459-4557-9188-8803bb3451fb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.648931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1288/2594] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:21.652578 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.665055 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871cc4f-a71d-4335-b1cc-ee8cc6a1cbeb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.673043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.673977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.674371 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.675019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] PATH_INFO: `/v3/projects/82da8f218ded48e5a6679466f69d1e0f/users/ac5b8f1809eb4f9bae7ce3ce1494def3/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.699021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82da8f218ded48e5a6679466f69d1e0f, user_id=ac5b8f1809eb4f9bae7ce3ce1494def3, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.701777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.716677 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:21.726582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1307/2595] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:21 2022] PUT /identity/v3/projects/82da8f218ded48e5a6679466f69d1e0f/users/ac5b8f1809eb4f9bae7ce3ce1494def3/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:21.743648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.744058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.744410 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.754434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:21.785370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] MFA Rules not processed for user `ac5b8f1809eb4f9bae7ce3ce1494def3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:21.786619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:21.866965 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1289/2596] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:21.871315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.944036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ac7e3-b1da-4e94-b492-3783342a64fc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:21.950248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:21.951148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:21.951499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:21.951840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:21.952823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:21.955802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:21.991947 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:21.992848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1308/2597] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:22.004951 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268ba2-2139-4317-a8ce-5399b44b90ed None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.074144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.074929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.075280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.075798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.076632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.079290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.103710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1290/2598] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:21 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:22.107560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:22.118592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026b376-6e7c-43e0-a08f-3318208f88e8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.123302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.123972 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.124257 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.125165 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.125910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.127564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.139515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1309/2599] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:22.144175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:22.157075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4515a7-361b-4a89-9e5f-36ba7947007f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.161931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.163081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.163460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.163747 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] PATH_INFO: `/v3/projects/023e75671b674a8cb37ff0ffafeacbbc/users/d05c5e2aed3c49e589516e64d734df4c/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.182818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=023e75671b674a8cb37ff0ffafeacbbc, user_id=d05c5e2aed3c49e589516e64d734df4c, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.185619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.205705 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1291/2600] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:22 2022] PUT /identity/v3/projects/023e75671b674a8cb37ff0ffafeacbbc/users/d05c5e2aed3c49e589516e64d734df4c/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:22.209583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:22.220365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c3568-8e96-4c09-a147-070f103b23db admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.284129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796ac1b-6009-4ecc-a837-086f2dff3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.284802 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ac1b-6009-4ecc-a837-086f2dff3115 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.285132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ac1b-6009-4ecc-a837-086f2dff3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.285440 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ac1b-6009-4ecc-a837-086f2dff3115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.286299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ac1b-6009-4ecc-a837-086f2dff3115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.288234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ac1b-6009-4ecc-a837-086f2dff3115 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.299445 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1310/2601] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:22.303510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:22.314565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c4d3-192e-42fb-baa1-909f1e35ff11 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.319342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.320233 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.320513 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.320785 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] PATH_INFO: `/v3/projects/023e75671b674a8cb37ff0ffafeacbbc/users/d05c5e2aed3c49e589516e64d734df4c/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.322791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=023e75671b674a8cb37ff0ffafeacbbc, user_id=d05c5e2aed3c49e589516e64d734df4c, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.325545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.335897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1292/2602] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:22 2022] PUT /identity/v3/projects/023e75671b674a8cb37ff0ffafeacbbc/users/d05c5e2aed3c49e589516e64d734df4c/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:22.347220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1797aa8-835f-472f-89ba-4ab1866f0d4b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.347513 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1797aa8-835f-472f-89ba-4ab1866f0d4b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.347796 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1797aa8-835f-472f-89ba-4ab1866f0d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.376056 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d1797aa8-835f-472f-89ba-4ab1866f0d4b None None] MFA Rules not processed for user `d05c5e2aed3c49e589516e64d734df4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:22.377331 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1797aa8-835f-472f-89ba-4ab1866f0d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:22.441805 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:22.453986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1311/2603] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:22.465993 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:22.470221 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:22.525599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.525917 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.526539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.547555 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] MFA Rules not processed for user `90dedb6ffa9f40fe905f60b023aeb92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:22.548644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:22.548970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:22.556480 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc01be-f9e2-4d63-bdcf-54d83443a80a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.560153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.560807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.561127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.561404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.572944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.574424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.585136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1293/2604] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:22.609916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1312/2605] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:22.667408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:22.677268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee17f1-b511-49aa-bd0a-8c8ab5b3ee63 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:22.683145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2cd0-6fe4-4b70-92fe-1dbf30dd1ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:22.683952 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2cd0-6fe4-4b70-92fe-1dbf30dd1ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:22.684367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2cd0-6fe4-4b70-92fe-1dbf30dd1ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:22.684772 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2cd0-6fe4-4b70-92fe-1dbf30dd1ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:22.757669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2cd0-6fe4-4b70-92fe-1dbf30dd1ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:22.759127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2cd0-6fe4-4b70-92fe-1dbf30dd1ee7 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:22.772013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1294/2606] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:23.102094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:23.171297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0de392-92cd-4440-bee1-09c2fd4cd6a4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:23.176581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:23.177671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:23.179153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:23.179153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] PATH_INFO: `/v3/users/ac5b8f1809eb4f9bae7ce3ce1494def3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:23.179789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5b8f1809eb4f9bae7ce3ce1494def3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:23.182003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:23.256776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5b8f1809eb4f9bae7ce3ce1494def3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:23.266682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5b8f1809eb4f9bae7ce3ce1494def3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:23.280554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5b8f1809eb4f9bae7ce3ce1494def3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:23.281852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5b8f1809eb4f9bae7ce3ce1494def3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:23.292022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5b8f1809eb4f9bae7ce3ce1494def3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:23.299552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1313/2607] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:23 2022] DELETE /identity/v3/users/ac5b8f1809eb4f9bae7ce3ce1494def3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:23.304241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:23.304922 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:23.305474 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:23.329227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:23.331160 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:23.400527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1295/2608] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:23.411685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:23.413629 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:23.443102 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:23.490699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa36d-46ca-453d-b23d-4ccee7e59bd0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:23.495035 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:23.495957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:23.495957 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:23.496240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:23.508969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:23.511025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:23.525086 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1314/2609] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:23.899292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:23.901677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:23.925314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fabb7-1e1c-431e-ae0a-e09df93bab5e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:23.933013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:23.934445 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:23.935074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:23.935521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] PATH_INFO: `/v3/projects/82da8f218ded48e5a6679466f69d1e0f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:23.938182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=82da8f218ded48e5a6679466f69d1e0f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:23.940409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:24.011433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82da8f218ded48e5a6679466f69d1e0f', 'request_id': 'req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.019434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82da8f218ded48e5a6679466f69d1e0f', 'request_id': 'req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.031821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82da8f218ded48e5a6679466f69d1e0f', 'request_id': 'req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.046355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] The token cache is being invalidate because project 82da8f218ded48e5a6679466f69d1e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:24.047387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.048164 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.053249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1296/2610] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:23 2022] DELETE /identity/v3/projects/82da8f218ded48e5a6679466f69d1e0f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:24.054557 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:24.067532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180efe-e906-4455-a6ca-9064cdab80a6 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:24.160009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:24.161259 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:24.161644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:24.162022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] PATH_INFO: `/v3/users/d05c5e2aed3c49e589516e64d734df4c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:24.163718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d05c5e2aed3c49e589516e64d734df4c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:24.165986 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:24.224156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd05c5e2aed3c49e589516e64d734df4c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.232854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd05c5e2aed3c49e589516e64d734df4c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.247101 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd05c5e2aed3c49e589516e64d734df4c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.247648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd05c5e2aed3c49e589516e64d734df4c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.256538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd05c5e2aed3c49e589516e64d734df4c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.268928 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1315/2611] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:24 2022] DELETE /identity/v3/users/d05c5e2aed3c49e589516e64d734df4c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:24.375657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:24.378456 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:24.396946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:24.482685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a29f3-2f7b-46da-a2e9-519fe47ca6fc admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:24.570155 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:24.570840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:24.571133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:24.571417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:24.575621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:24.590127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:24.590865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8eece-d2f1-4c4a-9ba1-0909ae7d9caa service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:24.602514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1297/2612] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:24.665358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:24.666860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:24.723523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:24.757308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7adbdf-6c23-4b59-ae1a-0d96bb514a4f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:24.764569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:24.766088 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:24.766718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:24.768996 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] PATH_INFO: `/v3/projects/023e75671b674a8cb37ff0ffafeacbbc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:24.769247 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=023e75671b674a8cb37ff0ffafeacbbc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:24.771604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:24.846474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '023e75671b674a8cb37ff0ffafeacbbc', 'request_id': 'req-e1995d49-bc05-461e-8d80-501399682510'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.853954 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '023e75671b674a8cb37ff0ffafeacbbc', 'request_id': 'req-e1995d49-bc05-461e-8d80-501399682510'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.867948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '023e75671b674a8cb37ff0ffafeacbbc', 'request_id': 'req-e1995d49-bc05-461e-8d80-501399682510'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.898395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] The token cache is being invalidate because project 023e75671b674a8cb37ff0ffafeacbbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:24.898701 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.899382 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1995d49-bc05-461e-8d80-501399682510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:24.902585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1316/2613] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:24 2022] DELETE /identity/v3/projects/023e75671b674a8cb37ff0ffafeacbbc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:24.910946 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:24.911902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:24.912703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:24.941693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:24.943505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:25.051597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1298/2614] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:25.058808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.061252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.061622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.082310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:25.083552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:25.092359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1317/2615] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:25.097154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.107283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c8657-48a4-4bed-b5d2-7dec5e361275 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.112365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.113083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.113371 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.113650 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.114390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:25.114704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.116370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.133527 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1299/2616] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:25.134310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.146121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d8630-f2c6-4343-82f3-6f42e78ab880 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.151433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.152395 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.152844 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.153152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.153857 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.156391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.193210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1318/2617] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:25 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:25.195315 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.209388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508695c-417a-4474-b29c-6e32bf94ac7f admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.245281 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=100 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:25.283595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.283948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.284295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.284672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.285478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.287899 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.311745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1300/2618] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:25 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:25.316983 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.329431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b96e07-f08b-4107-8b79-00da5d445d20 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.335213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.336239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.336597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.337009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.337832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.339548 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.352175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1319/2619] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:25.356319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.367359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46affae4-ca8b-4d68-99f3-78948417311d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.372106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c8f5f-5ffc-497a-9fb1-6c75bedd01d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.372768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c8f5f-5ffc-497a-9fb1-6c75bedd01d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.373081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c8f5f-5ffc-497a-9fb1-6c75bedd01d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.373355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c8f5f-5ffc-497a-9fb1-6c75bedd01d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.374094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c8f5f-5ffc-497a-9fb1-6c75bedd01d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.375734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c8f5f-5ffc-497a-9fb1-6c75bedd01d0 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.388907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1301/2620] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:25.393187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.403847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf306599-3248-4486-a085-6a5276d8f966 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.409300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.410190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.410470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.410728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] PATH_INFO: `/v3/projects/90c403c14db545b082dd1f4d68442ec6/users/1d8ec508a9c3448cae36e484f459af80/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.428430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c403c14db545b082dd1f4d68442ec6, user_id=1d8ec508a9c3448cae36e484f459af80, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.431173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.446753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1320/2621] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:25 2022] PUT /identity/v3/projects/90c403c14db545b082dd1f4d68442ec6/users/1d8ec508a9c3448cae36e484f459af80/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:25.458636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.458925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.459207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.492151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] MFA Rules not processed for user `1d8ec508a9c3448cae36e484f459af80`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:25.493597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:25.598558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1302/2622] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:25.613341 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:25.615065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:25.706173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbe582-41d1-46a4-8fea-801b458f78b1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:25.712553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd124d-7f11-40fd-9dd2-5e11a8cd8806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:25.713549 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd124d-7f11-40fd-9dd2-5e11a8cd8806 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:25.713887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd124d-7f11-40fd-9dd2-5e11a8cd8806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:25.714273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd124d-7f11-40fd-9dd2-5e11a8cd8806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:25.725828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd124d-7f11-40fd-9dd2-5e11a8cd8806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:25.726698 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd124d-7f11-40fd-9dd2-5e11a8cd8806 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:25.738966 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1321/2623] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:28.408105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:28.472118 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c5039-2130-4c5b-a4cc-9d4e54e06b79 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:28.476620 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:28.477522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:28.477810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:28.478083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] PATH_INFO: `/v3/users/1d8ec508a9c3448cae36e484f459af80` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:28.479302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d8ec508a9c3448cae36e484f459af80)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:28.481261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:28.526978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:28.533572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d8ec508a9c3448cae36e484f459af80'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:28.541811 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d8ec508a9c3448cae36e484f459af80'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:28.554320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d8ec508a9c3448cae36e484f459af80'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:28.554996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d8ec508a9c3448cae36e484f459af80'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:28.562408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d8ec508a9c3448cae36e484f459af80'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:28.568467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1303/2624] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:28 2022] DELETE /identity/v3/users/1d8ec508a9c3448cae36e484f459af80 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:28.572960 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:28.573463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:28.573575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:28.590561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:28.591779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:28.596671 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:28.663219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1322/2625] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:28.673587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:28.675022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:28.734416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936cbc7-1b61-4c7e-8888-af7031564917 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:28.738337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:28.739014 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:28.739323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:28.739596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:28.749138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:28.750610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:28.760290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1304/2626] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:29.125626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:29.127417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:29.139228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c0703-5125-4adf-b241-c6d1c0a8d408 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:29.144305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:29.145199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:29.145481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:29.146592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] PATH_INFO: `/v3/projects/90c403c14db545b082dd1f4d68442ec6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:29.147775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c403c14db545b082dd1f4d68442ec6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:29.149712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:29.201241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c403c14db545b082dd1f4d68442ec6', 'request_id': 'req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:29.208400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c403c14db545b082dd1f4d68442ec6', 'request_id': 'req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:29.217260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c403c14db545b082dd1f4d68442ec6', 'request_id': 'req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:29.229196 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] The token cache is being invalidate because project 90c403c14db545b082dd1f4d68442ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:29.229530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:29.230137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:29.232329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1323/2627] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:29 2022] DELETE /identity/v3/projects/90c403c14db545b082dd1f4d68442ec6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:38.755927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:39.226145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:43.859067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:43.861385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:43.874635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd7e6-d427-4997-82b5-5681d92fd790 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:43.924546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:43.925232 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:43.925518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:43.925821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:43.927625 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:43.992934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:43.994388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:44.005499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1305/2628] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:47.317191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:47.319010 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:47.340941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2ba8c-ddf0-4bf5-aead-d3be0a59ec88 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:47.346775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:47.348135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:47.348425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:47.348713 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] PATH_INFO: `/v3/users/c18c8d4034ce40dba9f9f48e96a19814` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:47.350531 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18c8d4034ce40dba9f9f48e96a19814)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:47.353995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:47.414449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18c8d4034ce40dba9f9f48e96a19814'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.423298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18c8d4034ce40dba9f9f48e96a19814'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.437774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18c8d4034ce40dba9f9f48e96a19814'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.439019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18c8d4034ce40dba9f9f48e96a19814'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.447747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e04e300-4670-478f-92ad-0d3accfa8c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18c8d4034ce40dba9f9f48e96a19814'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.459370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1324/2629] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:47 2022] DELETE /identity/v3/users/c18c8d4034ce40dba9f9f48e96a19814 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:47.687211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:47.705078 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cc80f-7028-41cb-99d9-3528ca873efd service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:47.780637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:47.782412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:47.782741 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:47.783025 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] PATH_INFO: `/v3/projects/ca9f11498f9d4245910281ac3bd55470` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:47.784269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9f11498f9d4245910281ac3bd55470)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:47.786213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:47.850942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9f11498f9d4245910281ac3bd55470', 'request_id': 'req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.854094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:47.860235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9f11498f9d4245910281ac3bd55470', 'request_id': 'req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.869796 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9f11498f9d4245910281ac3bd55470', 'request_id': 'req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.893727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] The token cache is being invalidate because project ca9f11498f9d4245910281ac3bd55470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:47.894038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.894762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-7d2bc7fc-8ff7-49c2-8650-b316eddc9c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:47.896424 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1306/2630] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:47 2022] DELETE /identity/v3/projects/ca9f11498f9d4245910281ac3bd55470 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:47.901859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:47.902948 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:47.903279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:47.929436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:47.930687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:48.002282 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1325/2631] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:48.013021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.013314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.013595 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.036301 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:48.038499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:48.048943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1307/2632] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:48.053287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.064745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574408f8-7c4e-4cee-a768-27ab048cf8e2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.071438 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.072491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.073013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.073468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.074691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:48.075199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.078534 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.093872 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1326/2633] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:48.098588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.108556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840da44-1d94-4602-99bd-93f934e472ee None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.114387 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.115055 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.115339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.115617 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.116302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.118251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.152942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1308/2634] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:48 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:48.154755 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.166404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e5bba-ee18-4aeb-9133-19d97a8a724d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.235164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.235840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.236127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.236405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.237121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.239034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.245763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:48.263639 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1327/2635] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:48 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:48.269111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.282596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac1719-d951-432e-b900-f2df489a2c42 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.289501 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.290111 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.290394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.290679 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.291409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.293054 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.305287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1309/2636] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:48.309502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.320360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58023e-db23-4a35-8856-e3087abc2577 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.328505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb33aa9-a145-4af2-9799-fae3a73d42b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.329672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33aa9-a145-4af2-9799-fae3a73d42b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.330119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33aa9-a145-4af2-9799-fae3a73d42b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.330581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33aa9-a145-4af2-9799-fae3a73d42b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.331896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb33aa9-a145-4af2-9799-fae3a73d42b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.334876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb33aa9-a145-4af2-9799-fae3a73d42b5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.352398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1328/2637] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:48.357119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.369773 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3d347-fc06-49c6-88fe-c7cc491ccdc4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.374462 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.375992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.376266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.376640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] PATH_INFO: `/v3/projects/a3bf27bef59e456db485599123591827/users/54e6a84c620541e2a54b4db93a104e07/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.395605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bf27bef59e456db485599123591827, user_id=54e6a84c620541e2a54b4db93a104e07, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.398312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.414531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1310/2638] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:48 2022] PUT /identity/v3/projects/a3bf27bef59e456db485599123591827/users/54e6a84c620541e2a54b4db93a104e07/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:48.428771 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.429446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.429446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.459359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] MFA Rules not processed for user `54e6a84c620541e2a54b4db93a104e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:48.460588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:48.545775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1329/2639] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:48.552392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.639651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dcae7-525e-4337-aedb-3b6538d6f9af admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.644689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.645401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.645688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.645982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.646787 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.648760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.680701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1311/2640] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:48 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:48.686248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.699922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818cb67-6fee-4b96-927c-9e7284a5247d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.768676 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.769933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.770227 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.770571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.771918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.773930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.798795 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1330/2641] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:48 2022] POST /identity/v3/users => generated 309 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:48.803690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.815496 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603c32-627a-45c8-bf53-255bba40dc79 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.820389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.821336 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.821644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.822057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.822804 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.824734 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.836917 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1312/2642] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:48.841485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.852329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961efd-782d-4d5c-98d8-8c06fdb4047e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.858515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.860017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.860302 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.860688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] PATH_INFO: `/v3/projects/d9ea13c9b7db42cab08570c74af7ed4b/users/c57fa79fc3c040628a9c5954953248db/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.876425 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ea13c9b7db42cab08570c74af7ed4b, user_id=c57fa79fc3c040628a9c5954953248db, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.879109 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.892648 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1331/2643] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:48 2022] PUT /identity/v3/projects/d9ea13c9b7db42cab08570c74af7ed4b/users/c57fa79fc3c040628a9c5954953248db/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:48.896781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:48.910119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e3c60-674a-4243-8594-ccae56b71d09 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:48.961682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:48.980413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f941e-cfa7-4187-980a-3acace3e2af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:48.981133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f941e-cfa7-4187-980a-3acace3e2af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:48.981425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f941e-cfa7-4187-980a-3acace3e2af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:48.981709 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f941e-cfa7-4187-980a-3acace3e2af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:48.982885 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f941e-cfa7-4187-980a-3acace3e2af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:48.985201 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f941e-cfa7-4187-980a-3acace3e2af2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:48.995750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1313/2644] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:49.000806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:49.010944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb3fce-029a-477b-bbb1-705b6b6beb23 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:49.015834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:49.016714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.017025 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.017304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] PATH_INFO: `/v3/projects/d9ea13c9b7db42cab08570c74af7ed4b/users/c57fa79fc3c040628a9c5954953248db/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.019252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ea13c9b7db42cab08570c74af7ed4b, user_id=c57fa79fc3c040628a9c5954953248db, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:49.021889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:49.032790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1332/2645] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:48 2022] PUT /identity/v3/projects/d9ea13c9b7db42cab08570c74af7ed4b/users/c57fa79fc3c040628a9c5954953248db/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:49.043346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.043645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.043927 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.073173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] MFA Rules not processed for user `c57fa79fc3c040628a9c5954953248db`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:49.074296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:49.142100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1314/2646] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:49.153040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:49.154815 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:49.178455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:49.218335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a95d8b-0396-493b-8d80-11bf44ee2fb1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:49.222628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd1b9b-c6c8-43b1-8f3e-948b4f08ce98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:49.223297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd1b9b-c6c8-43b1-8f3e-948b4f08ce98 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.223611 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd1b9b-c6c8-43b1-8f3e-948b4f08ce98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.223859 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd1b9b-c6c8-43b1-8f3e-948b4f08ce98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.234853 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd1b9b-c6c8-43b1-8f3e-948b4f08ce98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:49.236318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd1b9b-c6c8-43b1-8f3e-948b4f08ce98 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:49.246977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1333/2647] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:49 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:49.415807 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:49.478355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2d6d9-445e-4bb4-a4f9-ac13422fb005 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:49.483079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:49.484072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.484905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.485212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] PATH_INFO: `/v3/users/54e6a84c620541e2a54b4db93a104e07` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.486392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e6a84c620541e2a54b4db93a104e07)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:49.488314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:49.537585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e6a84c620541e2a54b4db93a104e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:49.544716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e6a84c620541e2a54b4db93a104e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:49.558005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e6a84c620541e2a54b4db93a104e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:49.558726 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e6a84c620541e2a54b4db93a104e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:49.560452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:49.567750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e6a84c620541e2a54b4db93a104e07'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:49.575502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1315/2648] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:49 2022] DELETE /identity/v3/users/54e6a84c620541e2a54b4db93a104e07 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:49.582398 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.582949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.582949 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.602900 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:49.604128 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:49.663706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1334/2649] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:49.674341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:49.675784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:49.737453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619c60c-b74e-4808-a2cf-9f2e16287136 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:49.741287 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:49.741933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.742217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.742494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.752098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:49.753567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:49.763861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1316/2650] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:50.054503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:50.056073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:50.070363 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7985222-bd5c-46c6-8ada-0677f80e05c3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:50.075301 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:50.076169 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:50.076446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:50.076735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] PATH_INFO: `/v3/projects/a3bf27bef59e456db485599123591827` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:50.078706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bf27bef59e456db485599123591827)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:50.080803 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:50.129141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bf27bef59e456db485599123591827', 'request_id': 'req-3d460752-d443-41c1-91fe-821c8dad3e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.134571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bf27bef59e456db485599123591827', 'request_id': 'req-3d460752-d443-41c1-91fe-821c8dad3e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.142418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bf27bef59e456db485599123591827', 'request_id': 'req-3d460752-d443-41c1-91fe-821c8dad3e25'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.155015 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] The token cache is being invalidate because project a3bf27bef59e456db485599123591827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:50.155304 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.156003 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.157593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1335/2651] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:50 2022] DELETE /identity/v3/projects/a3bf27bef59e456db485599123591827 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:50.162269 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:50.172489 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda6788-c310-4dc3-adcd-def14443cc3a service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:50.229986 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:50.230861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:50.231142 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:50.231423 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] PATH_INFO: `/v3/users/c57fa79fc3c040628a9c5954953248db` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:50.232605 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57fa79fc3c040628a9c5954953248db)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:50.234938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:50.281706 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57fa79fc3c040628a9c5954953248db'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.288758 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57fa79fc3c040628a9c5954953248db'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.300767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57fa79fc3c040628a9c5954953248db'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.301479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57fa79fc3c040628a9c5954953248db'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.309139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ebb266b7-702b-43dc-9d6a-0995f405a75b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57fa79fc3c040628a9c5954953248db'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.316549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1317/2652] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:50 2022] DELETE /identity/v3/users/c57fa79fc3c040628a9c5954953248db => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:50.748804 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:50.750808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:50.766185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460752-d443-41c1-91fe-821c8dad3e25 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:50.815156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:50.830870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:50.831762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:50.832027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:50.832300 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] PATH_INFO: `/v3/projects/d9ea13c9b7db42cab08570c74af7ed4b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:50.833499 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ea13c9b7db42cab08570c74af7ed4b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:50.835410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:50.888645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ea13c9b7db42cab08570c74af7ed4b', 'request_id': 'req-0f1a6601-2b38-4ea9-892d-fc99a071fa62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.895299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ea13c9b7db42cab08570c74af7ed4b', 'request_id': 'req-0f1a6601-2b38-4ea9-892d-fc99a071fa62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.904656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ea13c9b7db42cab08570c74af7ed4b', 'request_id': 'req-0f1a6601-2b38-4ea9-892d-fc99a071fa62'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.918110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] The token cache is being invalidate because project d9ea13c9b7db42cab08570c74af7ed4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:50.918501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.919241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0f1a6601-2b38-4ea9-892d-fc99a071fa62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:50.920863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1336/2653] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:50 2022] DELETE /identity/v3/projects/d9ea13c9b7db42cab08570c74af7ed4b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:50.926127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:50.926402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:50.926683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:50.945932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:50.947158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:50.958130 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:51.028660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1318/2654] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:51.038017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.038318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.038575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.059933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:51.061207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:51.070333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1337/2655] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:51.074528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.085623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9780c0-ff09-4b34-a20d-554983c3e094 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.090612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.091277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.091562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.091836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.092566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:51.092872 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.095257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.111360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1319/2656] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:51.117705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.128461 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f88e48-74e0-40d2-bb78-ea3fb4186eca None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.137180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.137180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.137180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.137180 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.137920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.137920 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.165229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1338/2657] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:51.169573 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.181058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b44163-eb56-45da-967c-4d08f74bd8b9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.241367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.241962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.242249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.242528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.243235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.245180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.266742 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1320/2658] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:51.270943 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.288464 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576401d5-6b89-4676-a7dd-e3f9bd595590 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.293570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.294239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.294520 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.294792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.295523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.297672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.309130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1339/2659] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:51.313956 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.324433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2d1c1-9f9f-4916-89f1-70b7853cf2a9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.329597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe715b25-f4bc-4d8b-ae0a-1e55473b82d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.330340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe715b25-f4bc-4d8b-ae0a-1e55473b82d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.330629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe715b25-f4bc-4d8b-ae0a-1e55473b82d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.330909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe715b25-f4bc-4d8b-ae0a-1e55473b82d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.331635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe715b25-f4bc-4d8b-ae0a-1e55473b82d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.333273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe715b25-f4bc-4d8b-ae0a-1e55473b82d4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.343247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1321/2660] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:51.348087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.358190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624e802-2c90-4589-8d65-ffe9ce89489a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.363420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.364326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.364609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.364906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] PATH_INFO: `/v3/projects/96a0079c9a83466aa74dab24b3a0ce5f/users/8da99c2f476143f2810f2ec12718ef2b/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.380749 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a0079c9a83466aa74dab24b3a0ce5f, user_id=8da99c2f476143f2810f2ec12718ef2b, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.383434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.396200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1340/2661] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:51 2022] PUT /identity/v3/projects/96a0079c9a83466aa74dab24b3a0ce5f/users/8da99c2f476143f2810f2ec12718ef2b/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:51.408260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.408550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.408836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.436146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] MFA Rules not processed for user `8da99c2f476143f2810f2ec12718ef2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:51.438631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:51.502716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1322/2662] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:51.507735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.576929 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc94a9-0efc-43c6-bbac-ea98959f02e7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.582218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.582879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.583166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.583443 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.584141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.586097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.605734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:51.614556 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1341/2663] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:51.619817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.631383 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba690d2-16f3-47d8-8038-19caf6c1e6ab None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.636217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:51.693112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77275b-6472-422b-b055-785183412236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.693792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77275b-6472-422b-b055-785183412236 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.694079 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77275b-6472-422b-b055-785183412236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.694361 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77275b-6472-422b-b055-785183412236 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.695074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77275b-6472-422b-b055-785183412236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.697094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77275b-6472-422b-b055-785183412236 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.718921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1323/2664] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:36:51.723027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.733410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928c940-7dc0-438e-abf0-c571345e69dd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.739034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.739830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.740153 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.740468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.741313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.743166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.753937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1342/2665] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:51.758125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77275b-6472-422b-b055-785183412236 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.768858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77275b-6472-422b-b055-785183412236 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.773672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.774557 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.774843 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.775119 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] PATH_INFO: `/v3/projects/157d74ed404b452bbe192791df549963/users/a57cd377a92741bf98abd467eae01cde/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.790828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d74ed404b452bbe192791df549963, user_id=a57cd377a92741bf98abd467eae01cde, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.793670 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.807228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1324/2666] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:51 2022] PUT /identity/v3/projects/157d74ed404b452bbe192791df549963/users/a57cd377a92741bf98abd467eae01cde/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:36:51.811189 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.822174 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f432f8-d87f-4f27-b954-4fabfd2b3666 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.882220 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546641e8-39ef-4f06-bdd7-a2296cd9939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.883201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546641e8-39ef-4f06-bdd7-a2296cd9939f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.883485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546641e8-39ef-4f06-bdd7-a2296cd9939f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.883762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546641e8-39ef-4f06-bdd7-a2296cd9939f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.884485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546641e8-39ef-4f06-bdd7-a2296cd9939f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.886126 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546641e8-39ef-4f06-bdd7-a2296cd9939f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.898325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1343/2667] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:51.902697 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:51.914213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18901cf-3a11-42b2-bf39-1524e778df08 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:51.919339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:51.920238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.920525 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.920801 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] PATH_INFO: `/v3/projects/157d74ed404b452bbe192791df549963/users/a57cd377a92741bf98abd467eae01cde/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.923664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d74ed404b452bbe192791df549963, user_id=a57cd377a92741bf98abd467eae01cde, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:51.926515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:51.936940 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1325/2668] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:51 2022] PUT /identity/v3/projects/157d74ed404b452bbe192791df549963/users/a57cd377a92741bf98abd467eae01cde/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:51.946824 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:51.947114 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:51.947401 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:51.977740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] MFA Rules not processed for user `a57cd377a92741bf98abd467eae01cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:51.978963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:52.047303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1344/2669] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:52.058242 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:52.059934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:52.117594 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba3a4-84ec-49f4-8c9e-67adcece4fd8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:52.121283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:52.121930 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:52.122214 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:52.122488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:52.132981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:52.134487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:52.146102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1326/2670] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:52.193333 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:52.194721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:52.206893 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3678-8c0d-4cd6-9c1d-9ef1e541e85d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:52.210467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d1bcf-665e-4ce6-98cd-9ba4c21e2219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:52.211138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1bcf-665e-4ce6-98cd-9ba4c21e2219 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:52.211418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1bcf-665e-4ce6-98cd-9ba4c21e2219 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:52.211690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1bcf-665e-4ce6-98cd-9ba4c21e2219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:52.269801 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d1bcf-665e-4ce6-98cd-9ba4c21e2219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:52.271270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d1bcf-665e-4ce6-98cd-9ba4c21e2219 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:52.280516 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1345/2671] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:55.593211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:55.655056 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7f475-ea7a-40d4-86fd-867d435d58e4 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:55.659767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:55.660635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:55.660943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:55.661222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] PATH_INFO: `/v3/users/8da99c2f476143f2810f2ec12718ef2b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:55.662405 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da99c2f476143f2810f2ec12718ef2b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:55.664606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:55.669362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:55.718394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da99c2f476143f2810f2ec12718ef2b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:55.727632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da99c2f476143f2810f2ec12718ef2b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:55.740345 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da99c2f476143f2810f2ec12718ef2b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:55.741342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da99c2f476143f2810f2ec12718ef2b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:55.748688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da99c2f476143f2810f2ec12718ef2b'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:55.755609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1327/2672] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:55 2022] DELETE /identity/v3/users/8da99c2f476143f2810f2ec12718ef2b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:55.760337 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:55.761162 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:55.761466 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:55.774207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:55.780609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:55.782060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:55.841219 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1346/2673] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:55.851206 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:55.852618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:55.911295 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f59f57-1c64-462a-94ec-f92e93ee191a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:55.915222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:55.915869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:55.916178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:55.916460 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:55.926398 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:55.927832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:55.937833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1328/2674] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:56.218355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:56.220325 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:56.232540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612b2e8-6dd2-4e8b-a5a4-2c5cab400ea1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:56.238064 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:56.238967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.239256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.239541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] PATH_INFO: `/v3/projects/96a0079c9a83466aa74dab24b3a0ce5f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.240741 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a0079c9a83466aa74dab24b3a0ce5f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:56.242711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:56.292982 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a0079c9a83466aa74dab24b3a0ce5f', 'request_id': 'req-f567a1a0-9f3e-49b5-9c99-b033b70ac223'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.301439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a0079c9a83466aa74dab24b3a0ce5f', 'request_id': 'req-f567a1a0-9f3e-49b5-9c99-b033b70ac223'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.309670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a0079c9a83466aa74dab24b3a0ce5f', 'request_id': 'req-f567a1a0-9f3e-49b5-9c99-b033b70ac223'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.323027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] The token cache is being invalidate because project 96a0079c9a83466aa74dab24b3a0ce5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:56.323319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.323965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.325560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1347/2675] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:56 2022] DELETE /identity/v3/projects/96a0079c9a83466aa74dab24b3a0ce5f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:56.334426 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:56.344792 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e79511-5c5e-4be0-b43f-c47d63eb7c44 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:56.403235 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:56.404157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.404469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.404750 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] PATH_INFO: `/v3/users/a57cd377a92741bf98abd467eae01cde` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.405991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57cd377a92741bf98abd467eae01cde)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:56.408837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:56.437646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:56.459611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57cd377a92741bf98abd467eae01cde'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.466962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57cd377a92741bf98abd467eae01cde'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.484299 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57cd377a92741bf98abd467eae01cde'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.485154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57cd377a92741bf98abd467eae01cde'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.493832 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f36d0633-fcf3-4eb1-b020-abad2e030bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57cd377a92741bf98abd467eae01cde'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.500122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1329/2676] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:36:56 2022] DELETE /identity/v3/users/a57cd377a92741bf98abd467eae01cde => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:56.792491 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:56.797295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:56.812580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a1a0-9f3e-49b5-9c99-b033b70ac223 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:56.874832 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:56.876161 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.876485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.876769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] PATH_INFO: `/v3/projects/157d74ed404b452bbe192791df549963` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.878184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157d74ed404b452bbe192791df549963)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:56.880108 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:56.922260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:56.936940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157d74ed404b452bbe192791df549963', 'request_id': 'req-3bc314b5-1ead-4151-b1d4-ae1373891d06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.942970 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157d74ed404b452bbe192791df549963', 'request_id': 'req-3bc314b5-1ead-4151-b1d4-ae1373891d06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.951428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157d74ed404b452bbe192791df549963', 'request_id': 'req-3bc314b5-1ead-4151-b1d4-ae1373891d06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.965423 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] The token cache is being invalidate because project 157d74ed404b452bbe192791df549963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:36:56.965887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.966799 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3bc314b5-1ead-4151-b1d4-ae1373891d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:36:56.968408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1348/2677] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:36:56 2022] DELETE /identity/v3/projects/157d74ed404b452bbe192791df549963 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:56.974414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.974924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.975058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.994810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:56.996030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.061543 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1330/2678] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:36:57.070540 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.070830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.071107 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.091131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:57.092394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.100632 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1349/2679] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:57.107135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.118541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4a2fa-1b9a-4d61-82da-b936e3055060 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.123320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.123996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.124284 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.124560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.125931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:36:57.126380 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.128262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.140614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1331/2680] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:36:57.145817 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.155198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42561b9-cad7-40b5-b371-c5b73b82d157 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.160475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.161191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.161485 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.161761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.162467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.164618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.194397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1350/2681] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:36:57 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:57.198653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.210105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9c587-476f-4c16-8dfa-f71ff2fd82a7 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.276440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.277153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.277440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.277719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.278439 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.280360 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.301835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1332/2682] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:36:57 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:36:57.307937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.319644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3bef12-7036-4b3d-a57b-cd13464b5cb9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.324695 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.325400 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.325687 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.325969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.326717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.328328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.339745 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1351/2683] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:57.343945 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.354351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6929c5a-26ae-4bfa-bebb-93fa2c9094b1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.361179 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb9aa1-b999-42cb-b42f-e952a470ef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.362083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb9aa1-b999-42cb-b42f-e952a470ef62 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.362378 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb9aa1-b999-42cb-b42f-e952a470ef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.362662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb9aa1-b999-42cb-b42f-e952a470ef62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.363438 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb9aa1-b999-42cb-b42f-e952a470ef62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.365113 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb9aa1-b999-42cb-b42f-e952a470ef62 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.378370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1333/2684] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:36:57.382977 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.393602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f65aa-d1f6-484c-84dd-06e093b0f8d7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.398406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.399311 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.399599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.399874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] PATH_INFO: `/v3/projects/d974dc4f2d4c405cad746422c69c5887/users/149bde065e6a4fb68f649363639e6293/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.416386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d974dc4f2d4c405cad746422c69c5887, user_id=149bde065e6a4fb68f649363639e6293, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.419091 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.433429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1352/2685] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:36:57 2022] PUT /identity/v3/projects/d974dc4f2d4c405cad746422c69c5887/users/149bde065e6a4fb68f649363639e6293/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:36:57.444948 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.445263 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.445548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.465436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:57.474061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] MFA Rules not processed for user `149bde065e6a4fb68f649363639e6293`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:36:57.476083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.552124 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1334/2686] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:36:57.564391 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:57.566248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:57.626139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa7398-95f0-40f5-91d2-d7ec41e2fda9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:57.630320 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714662f2-00b9-4e2c-947e-c2ab1c01025c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:57.630911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714662f2-00b9-4e2c-947e-c2ab1c01025c service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:57.631199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714662f2-00b9-4e2c-947e-c2ab1c01025c service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:57.631479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714662f2-00b9-4e2c-947e-c2ab1c01025c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:57.640665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714662f2-00b9-4e2c-947e-c2ab1c01025c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:57.642848 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714662f2-00b9-4e2c-947e-c2ab1c01025c service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:57.652430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1353/2687] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:07.548908 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=27 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:07.648111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:16.274728 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:16.339596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685310c-2fc4-4077-84f6-eb1af9729041 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:16.344286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:16.345196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:16.345481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:16.345798 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] PATH_INFO: `/v3/users/149bde065e6a4fb68f649363639e6293` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:16.347673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] RBAC: Authorizing `identity:delete_user(user_id=149bde065e6a4fb68f649363639e6293)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:16.350862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:16.398249 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149bde065e6a4fb68f649363639e6293'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:16.405714 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149bde065e6a4fb68f649363639e6293'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:16.418481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149bde065e6a4fb68f649363639e6293'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:16.419180 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149bde065e6a4fb68f649363639e6293'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:16.426968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149bde065e6a4fb68f649363639e6293'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:16.433093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1335/2688] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:37:16 2022] DELETE /identity/v3/users/149bde065e6a4fb68f649363639e6293 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:16.437965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:16.438901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:16.438901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:16.457067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:16.458266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:16.527991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1354/2689] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:16.538319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:16.539784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:16.610317 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce5225-6158-41c3-8544-f5f47f8f709e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:16.614926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8e4fc-66f6-4d34-ab43-bdea8a3cad02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:16.615578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8e4fc-66f6-4d34-ab43-bdea8a3cad02 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:16.615861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8e4fc-66f6-4d34-ab43-bdea8a3cad02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:16.616139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8e4fc-66f6-4d34-ab43-bdea8a3cad02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:16.625516 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8e4fc-66f6-4d34-ab43-bdea8a3cad02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:16.627542 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8e4fc-66f6-4d34-ab43-bdea8a3cad02 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:16.637641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1336/2690] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:16.920489 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:16.922365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:16.938104 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c70ce-29c9-49f8-aa0e-85a17b87325a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:16.943779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:16.944901 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:16.945264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:16.946009 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] PATH_INFO: `/v3/projects/d974dc4f2d4c405cad746422c69c5887` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:16.946814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d974dc4f2d4c405cad746422c69c5887)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:16.948645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.004646 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd974dc4f2d4c405cad746422c69c5887', 'request_id': 'req-b728e93f-94bb-464c-b445-5c8816ce447e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:17.010356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd974dc4f2d4c405cad746422c69c5887', 'request_id': 'req-b728e93f-94bb-464c-b445-5c8816ce447e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:17.018201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd974dc4f2d4c405cad746422c69c5887', 'request_id': 'req-b728e93f-94bb-464c-b445-5c8816ce447e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:17.029568 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] The token cache is being invalidate because project d974dc4f2d4c405cad746422c69c5887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:37:17.029861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:17.030509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b728e93f-94bb-464c-b445-5c8816ce447e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:17.032112 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1355/2691] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:37:16 2022] DELETE /identity/v3/projects/d974dc4f2d4c405cad746422c69c5887 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:17.037678 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.038359 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.038744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.057435 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:17.058702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:17.076182 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:17.122870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1337/2692] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:17.131181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.131468 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.131756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.152049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:17.153511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:17.162573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1356/2693] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:17.167141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.178261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95876e00-0e89-4852-8cbb-ae5943931916 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.182915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.183583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.183870 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.184150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.184902 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:37:17.185222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.186856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.201046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1338/2694] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:37:17.205998 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.215721 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2876b2-2e7d-46bf-9159-13c32cb5a870 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.221307 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.221979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.222255 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.222526 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.223236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.225229 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.258432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1357/2695] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:37:17.260209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.271145 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73bfda-4f81-40cb-84a0-fd7f9188e053 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.329209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.329904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.330716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.331021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.331703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.333663 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.355865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1339/2696] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:17.360054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.371164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a194c3-943c-40f7-9824-dea147c26d71 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.376216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.377214 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.377538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.377812 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.378576 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.380198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.391359 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1358/2697] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:17.396454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.406766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826eb7a-21ae-4f04-8561-ddb7470d0401 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.412000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca69ba-a206-4a06-9e3c-012e5e743bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.412662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca69ba-a206-4a06-9e3c-012e5e743bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.412973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca69ba-a206-4a06-9e3c-012e5e743bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.413509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca69ba-a206-4a06-9e3c-012e5e743bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.414239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca69ba-a206-4a06-9e3c-012e5e743bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.415828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca69ba-a206-4a06-9e3c-012e5e743bfa admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.426260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1340/2698] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:17.430465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.440774 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4ec8-260d-45af-9728-f4e1778c9ef7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.445475 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.446353 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.446637 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.446911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] PATH_INFO: `/v3/projects/ad18bf48c7d74445823a93929fa8d291/users/dd554fee0af74230a756d075ad2c7632/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.462603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad18bf48c7d74445823a93929fa8d291, user_id=dd554fee0af74230a756d075ad2c7632, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.465429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.479642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1359/2699] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:17 2022] PUT /identity/v3/projects/ad18bf48c7d74445823a93929fa8d291/users/dd554fee0af74230a756d075ad2c7632/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:17.491067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.491367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.491652 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.519523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] MFA Rules not processed for user `dd554fee0af74230a756d075ad2c7632`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:17.521064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:17.589210 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1341/2700] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:17.593634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.615043 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:17.660682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f166c-dd4e-4885-ad69-98f3ef940b9f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.665995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.666660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.666947 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.667223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.668018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.670059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.696483 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1360/2701] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:17.700686 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.712454 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b045f7-c14a-4946-844c-0de4dbce6fee None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.725722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:17.773660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.774343 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.774636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.774915 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.775618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.777688 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.799905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1342/2702] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:17.804024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.815032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fc42f-7cba-4e72-92cc-9f108bbddffb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.819638 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.820292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.820579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.820847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.821601 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.823184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.835650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1361/2703] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:17.840244 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.855647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6245b-4795-4f20-a963-efd167fda69d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.862619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1abf3e-5de2-4bf9-90bd-ada28f570682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.863649 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1abf3e-5de2-4bf9-90bd-ada28f570682 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.864108 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1abf3e-5de2-4bf9-90bd-ada28f570682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.864535 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1abf3e-5de2-4bf9-90bd-ada28f570682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.865744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1abf3e-5de2-4bf9-90bd-ada28f570682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.868609 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1abf3e-5de2-4bf9-90bd-ada28f570682 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.885869 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1343/2704] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:17.890231 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:17.901479 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac57db-199d-4d0e-99e3-84decb7b25c7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:17.906975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:17.909152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.909152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.909152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] PATH_INFO: `/v3/projects/221d8ad7d7db4736a0946033150830da/users/84be52a31a3a4cd282e2c1c9d4c2304d/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.924850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221d8ad7d7db4736a0946033150830da, user_id=84be52a31a3a4cd282e2c1c9d4c2304d, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:17.927565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:17.941155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1362/2705] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:17 2022] PUT /identity/v3/projects/221d8ad7d7db4736a0946033150830da/users/84be52a31a3a4cd282e2c1c9d4c2304d/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:17.952030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:17.952470 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:17.952906 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:17.987884 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] MFA Rules not processed for user `84be52a31a3a4cd282e2c1c9d4c2304d`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:17.989224 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:18.060259 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1344/2706] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:18.064904 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.129415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d420bc-309e-4b3f-ae10-2ba78708d972 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.134096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.134787 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.135070 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.135347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.136042 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.137985 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.167462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1363/2707] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:37:18 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:18.171514 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.183316 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f16d-0abf-4c63-8342-fc78504b309a None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.241612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.242296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.242584 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.242863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.243556 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.245541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.265748 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1345/2708] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:37:18 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:18.270373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.280910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fac3be-bdea-4ed8-89c0-1204d4fdcb85 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.286560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.287269 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.287560 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.287842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.288846 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.290492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.302011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1364/2709] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:18.306344 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.316623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5c0fb-e34a-4c8f-b37a-1a089169f9f4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.321311 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.322193 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.322475 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.322745 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] PATH_INFO: `/v3/projects/e7a57eb1c6f94dffb93f8da49f2f9b4b/users/6e8eec8eb1c5485b9aaeb9c06294c680/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.336123 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:18.339506 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a57eb1c6f94dffb93f8da49f2f9b4b, user_id=6e8eec8eb1c5485b9aaeb9c06294c680, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.342502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.358644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1346/2710] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:18 2022] PUT /identity/v3/projects/e7a57eb1c6f94dffb93f8da49f2f9b4b/users/6e8eec8eb1c5485b9aaeb9c06294c680/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:18.363063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.373355 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876c765-b00b-4624-837a-21ebe2751589 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.418274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:18.434462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f163dbf-59bc-4ae9-83a3-dfc7ccc1d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.435144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f163dbf-59bc-4ae9-83a3-dfc7ccc1d55d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.435433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f163dbf-59bc-4ae9-83a3-dfc7ccc1d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.435715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f163dbf-59bc-4ae9-83a3-dfc7ccc1d55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.436456 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f163dbf-59bc-4ae9-83a3-dfc7ccc1d55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.438079 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f163dbf-59bc-4ae9-83a3-dfc7ccc1d55d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.448609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1365/2711] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:37:18.457407 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.467966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce291e-f822-473b-8570-b25b9b6dd5e6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.473260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.474158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.474441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.474715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] PATH_INFO: `/v3/projects/e7a57eb1c6f94dffb93f8da49f2f9b4b/users/6e8eec8eb1c5485b9aaeb9c06294c680/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.476721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a57eb1c6f94dffb93f8da49f2f9b4b, user_id=6e8eec8eb1c5485b9aaeb9c06294c680, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.479412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.490531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1347/2712] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:18 2022] PUT /identity/v3/projects/e7a57eb1c6f94dffb93f8da49f2f9b4b/users/6e8eec8eb1c5485b9aaeb9c06294c680/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:18.501058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.501347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.501622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.530509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] MFA Rules not processed for user `6e8eec8eb1c5485b9aaeb9c06294c680`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:18.531830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:18.606786 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1366/2713] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:18.620828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:18.624567 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:18.686752 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e79aa-bebd-4add-8471-64cd277a5938 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:18.690744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:18.691406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:18.691693 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:18.691976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:18.751069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:18.752608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:18.762328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1348/2714] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:20.263201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:20.264634 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:20.276964 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42b743-4d8d-4bb3-816a-64f18198b988 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:20.280521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:20.281204 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:20.281492 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:20.281769 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:20.346739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:20.348094 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:20.357178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1367/2715] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:20.992502 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:21.001509 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2042d51-eaf7-49ec-a78c-521abc645f3e service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:21.007218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33e8de-c146-4eb5-b00c-92cfe8e2b093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:21.008250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33e8de-c146-4eb5-b00c-92cfe8e2b093 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:21.008642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33e8de-c146-4eb5-b00c-92cfe8e2b093 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:21.009061 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33e8de-c146-4eb5-b00c-92cfe8e2b093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:21.020583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33e8de-c146-4eb5-b00c-92cfe8e2b093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:21.022185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33e8de-c146-4eb5-b00c-92cfe8e2b093 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:21.032157 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1349/2716] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:24.926800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:24.993447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc62d1-ac4e-4c0a-9f7c-3fd669001f1b service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:24.998207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:24.999111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:24.999396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:24.999673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] PATH_INFO: `/v3/users/dd554fee0af74230a756d075ad2c7632` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:25.000858 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd554fee0af74230a756d075ad2c7632)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:25.003221 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:25.007942 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:25.058610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd554fee0af74230a756d075ad2c7632'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.065936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd554fee0af74230a756d075ad2c7632'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.078085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd554fee0af74230a756d075ad2c7632'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.078783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd554fee0af74230a756d075ad2c7632'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.087008 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd554fee0af74230a756d075ad2c7632'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.093806 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1368/2717] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:37:24 2022] DELETE /identity/v3/users/dd554fee0af74230a756d075ad2c7632 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:25.099126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:25.099658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:25.099777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:25.116805 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:25.118655 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:25.187707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1350/2718] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:25.200328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:25.200328 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:25.265728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d61d9f-1fe7-4b2f-ada0-7707754c50d4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:25.271833 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:25.272757 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:25.273087 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:25.273367 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:25.283264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:25.284728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:25.296918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1369/2719] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:25.591651 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:25.592937 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:25.605465 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe904c-6659-4ac8-984f-44f44ea2d9d9 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:25.610283 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:25.611698 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:25.612092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:25.612389 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] PATH_INFO: `/v3/projects/ad18bf48c7d74445823a93929fa8d291` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:25.613641 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad18bf48c7d74445823a93929fa8d291)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:25.615637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:25.637428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:25.675158 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad18bf48c7d74445823a93929fa8d291', 'request_id': 'req-3a8274a4-b465-41bc-a561-4c3983eca0c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.682086 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad18bf48c7d74445823a93929fa8d291', 'request_id': 'req-3a8274a4-b465-41bc-a561-4c3983eca0c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.696009 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad18bf48c7d74445823a93929fa8d291', 'request_id': 'req-3a8274a4-b465-41bc-a561-4c3983eca0c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.708618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] The token cache is being invalidate because project ad18bf48c7d74445823a93929fa8d291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:37:25.708938 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.709664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.711268 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1351/2720] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:37:25 2022] DELETE /identity/v3/projects/ad18bf48c7d74445823a93929fa8d291 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:25.715191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:25.725547 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050f18-1bbb-4bfb-ad9e-9463b41d328e service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:25.785216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:25.786106 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:25.786447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:25.786691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] PATH_INFO: `/v3/users/84be52a31a3a4cd282e2c1c9d4c2304d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:25.788066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84be52a31a3a4cd282e2c1c9d4c2304d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:25.790298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:25.824166 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:25.843319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84be52a31a3a4cd282e2c1c9d4c2304d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.851115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84be52a31a3a4cd282e2c1c9d4c2304d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.864289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84be52a31a3a4cd282e2c1c9d4c2304d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.865072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84be52a31a3a4cd282e2c1c9d4c2304d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.873366 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84be52a31a3a4cd282e2c1c9d4c2304d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:25.880184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1370/2721] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:37:25 2022] DELETE /identity/v3/users/84be52a31a3a4cd282e2c1c9d4c2304d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:26.157277 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:26.159591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:26.175716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8274a4-b465-41bc-a561-4c3983eca0c3 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:26.235968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:26.237433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:26.237721 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:26.238001 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] PATH_INFO: `/v3/projects/221d8ad7d7db4736a0946033150830da` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:26.239198 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=221d8ad7d7db4736a0946033150830da)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:26.241172 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:26.286657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221d8ad7d7db4736a0946033150830da', 'request_id': 'req-b27cc6e6-67e4-414f-ae04-ffddf0426efb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.292531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221d8ad7d7db4736a0946033150830da', 'request_id': 'req-b27cc6e6-67e4-414f-ae04-ffddf0426efb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.300708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221d8ad7d7db4736a0946033150830da', 'request_id': 'req-b27cc6e6-67e4-414f-ae04-ffddf0426efb'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.311953 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] The token cache is being invalidate because project 221d8ad7d7db4736a0946033150830da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:37:26.312247 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.312904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.314664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1352/2722] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:37:26 2022] DELETE /identity/v3/projects/221d8ad7d7db4736a0946033150830da => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:26.318590 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:26.320017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:26.333916 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81d628-759f-4b77-87f7-eb114600bd67 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:26.391488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:26.392910 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:26.393199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:26.393559 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] PATH_INFO: `/v3/users/6e8eec8eb1c5485b9aaeb9c06294c680` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:26.394899 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8eec8eb1c5485b9aaeb9c06294c680)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:26.396813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:26.443394 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8eec8eb1c5485b9aaeb9c06294c680'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.450312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8eec8eb1c5485b9aaeb9c06294c680'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.462529 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8eec8eb1c5485b9aaeb9c06294c680'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.463198 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8eec8eb1c5485b9aaeb9c06294c680'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.471981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5adf478c-6b4e-4a66-867b-1a129ad68b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8eec8eb1c5485b9aaeb9c06294c680'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.478138 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1371/2723] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:37:26 2022] DELETE /identity/v3/users/6e8eec8eb1c5485b9aaeb9c06294c680 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:26.775437 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:26.776933 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:26.790309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cc6e6-67e4-414f-ae04-ffddf0426efb admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:26.825125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:26.855766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:26.856562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:26.856856 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:26.857173 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] PATH_INFO: `/v3/projects/e7a57eb1c6f94dffb93f8da49f2f9b4b` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:26.858394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a57eb1c6f94dffb93f8da49f2f9b4b)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:26.860385 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:26.912045 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a57eb1c6f94dffb93f8da49f2f9b4b', 'request_id': 'req-ae215c12-dc6e-4f6d-afb7-b44e427df180'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.917834 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a57eb1c6f94dffb93f8da49f2f9b4b', 'request_id': 'req-ae215c12-dc6e-4f6d-afb7-b44e427df180'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.925736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a57eb1c6f94dffb93f8da49f2f9b4b', 'request_id': 'req-ae215c12-dc6e-4f6d-afb7-b44e427df180'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.937701 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] The token cache is being invalidate because project e7a57eb1c6f94dffb93f8da49f2f9b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:37:26.937990 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.938640 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-ae215c12-dc6e-4f6d-afb7-b44e427df180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:26.940777 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1353/2724] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:37:26 2022] DELETE /identity/v3/projects/e7a57eb1c6f94dffb93f8da49f2f9b4b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:37:26.946501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:26.946791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:26.947067 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:26.965742 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:26.966979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:26.989670 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:27.039160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1372/2725] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:27.047393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.047685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.048019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.067005 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:27.068211 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:27.076231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1354/2726] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:27.081360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.091281 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ece0-7ea0-4e22-9d2e-0705c0525a9c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.096141 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.096903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.097200 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.097481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.098218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:37:27.098532 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.100148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.112509 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1373/2727] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:37:27.116806 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.126646 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6899c-abe2-4360-aa5e-32083abba9a7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.132285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.132983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.133270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.133539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.134349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.136519 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.165740 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1355/2728] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:37:27 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:37:27.167308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.178218 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b70f-c216-41d4-9b8a-d2b8a166c7ba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.234223 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.234946 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.235185 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.235465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.236663 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.238776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.261007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1374/2729] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:37:27 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:27.264408 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.275739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc0dd-15ad-4b8c-94e3-429a4b0e5063 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.281422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.282109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.282397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.282725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.283528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.285286 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.305813 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1356/2730] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:27.306239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.317652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182fb20-5b12-4bc4-be06-9adb73af9f99 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.322620 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4009e-fa4a-4cd7-9dab-9cf7f558cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.323299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4009e-fa4a-4cd7-9dab-9cf7f558cf20 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.323790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4009e-fa4a-4cd7-9dab-9cf7f558cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.324072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4009e-fa4a-4cd7-9dab-9cf7f558cf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.324810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4009e-fa4a-4cd7-9dab-9cf7f558cf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.326490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4009e-fa4a-4cd7-9dab-9cf7f558cf20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.337188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1375/2731] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:37:27.346333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.356842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678cd7-6611-42d6-ac04-05c8bd7dd96a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.362089 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.363018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.363275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.363550 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] PATH_INFO: `/v3/projects/aeac6317b2334152adb2b0b580f5b1a8/users/59e004e178b94b69ac4d108324da2cf3/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.380538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeac6317b2334152adb2b0b580f5b1a8, user_id=59e004e178b94b69ac4d108324da2cf3, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.383209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.396833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1357/2732] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:27 2022] PUT /identity/v3/projects/aeac6317b2334152adb2b0b580f5b1a8/users/59e004e178b94b69ac4d108324da2cf3/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:27.407191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1ac5c-7f2e-4051-9f0b-5dce9f69545c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.407480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1ac5c-7f2e-4051-9f0b-5dce9f69545c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.407763 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1ac5c-7f2e-4051-9f0b-5dce9f69545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.439248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a8c1ac5c-7f2e-4051-9f0b-5dce9f69545c None None] MFA Rules not processed for user `59e004e178b94b69ac4d108324da2cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:27.441792 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a8c1ac5c-7f2e-4051-9f0b-5dce9f69545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:27.512379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1376/2733] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:27.524562 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:27.526288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:27.584958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e91b65-a416-42ea-9987-1449ea88bcd1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:27.588619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:27.589296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:27.589613 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:27.589894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:27.599810 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:27.601432 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:27.610943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1358/2734] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:29.978244 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440eb58-1b2d-4981-80b2-139f67530bf7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:29.978762 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440eb58-1b2d-4981-80b2-139f67530bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:29.978902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440eb58-1b2d-4981-80b2-139f67530bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:29.996404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f440eb58-1b2d-4981-80b2-139f67530bf7 None None] MFA Rules not processed for user `90dedb6ffa9f40fe905f60b023aeb92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:29.997640 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f440eb58-1b2d-4981-80b2-139f67530bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:30.010968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:30.055279 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1377/2735] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:37.606762 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:37.720166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:37.782588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41709938-e8df-4d58-bf19-8e2c4b011334 service cinder] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:37.787673 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:37.788803 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:37.789208 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:37.789492 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] PATH_INFO: `/v3/users/59e004e178b94b69ac4d108324da2cf3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:37.790825 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e004e178b94b69ac4d108324da2cf3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:37.792744 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:37.846637 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e004e178b94b69ac4d108324da2cf3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:37.855352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e004e178b94b69ac4d108324da2cf3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:37.868544 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e004e178b94b69ac4d108324da2cf3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:37.869306 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e004e178b94b69ac4d108324da2cf3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:37.878006 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e004e178b94b69ac4d108324da2cf3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:37.884216 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1359/2736] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:37:37 2022] DELETE /identity/v3/users/59e004e178b94b69ac4d108324da2cf3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:37.888962 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:37.889441 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:37.889580 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:37.907604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:37.909054 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:37.969915 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1378/2737] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:37.980072 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:37.981538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:38.038631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d3583-9458-4d05-b941-d4fd3dec31be admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.042288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bae2d9-0ee2-49a5-adda-3479389f05f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.042932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bae2d9-0ee2-49a5-adda-3479389f05f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.043217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bae2d9-0ee2-49a5-adda-3479389f05f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.043490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bae2d9-0ee2-49a5-adda-3479389f05f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.053161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bae2d9-0ee2-49a5-adda-3479389f05f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.054597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bae2d9-0ee2-49a5-adda-3479389f05f7 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.064333 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1360/2738] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:38.334396 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.335875 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:38.349239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7abd-4967-40da-9070-f23cca566aaf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.353881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.354660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.355256 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.355830 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] PATH_INFO: `/v3/projects/aeac6317b2334152adb2b0b580f5b1a8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.357515 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeac6317b2334152adb2b0b580f5b1a8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.359422 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.412224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeac6317b2334152adb2b0b580f5b1a8', 'request_id': 'req-4523366b-6f75-4f02-b756-cbc59c8c6a7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:38.419155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeac6317b2334152adb2b0b580f5b1a8', 'request_id': 'req-4523366b-6f75-4f02-b756-cbc59c8c6a7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:38.429421 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeac6317b2334152adb2b0b580f5b1a8', 'request_id': 'req-4523366b-6f75-4f02-b756-cbc59c8c6a7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:38.441594 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] The token cache is being invalidate because project aeac6317b2334152adb2b0b580f5b1a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:37:38.441895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:38.442552 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4523366b-6f75-4f02-b756-cbc59c8c6a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:37:38.444155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1379/2739] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:37:38 2022] DELETE /identity/v3/projects/aeac6317b2334152adb2b0b580f5b1a8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:38.449303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.449627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.449901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.468230 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:38.469494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:38.488022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:38.545973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1361/2740] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:38.609390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.609682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.609965 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.631465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:38.632706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:38.643332 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1380/2741] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:37:38.647782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.658418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457831-5f7b-4a57-b1ca-1e7e3ef632fe None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.664007 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.664664 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.664981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.665261 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.666027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:37:38.666338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.667983 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.681467 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1362/2742] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:37:38.686019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.695723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb85ba7-0aee-4030-8361-76c2f4ee5146 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.700618 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.701323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.701616 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.701889 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.702585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.704525 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.744377 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1381/2743] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:37:38 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:37:38.748658 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.760630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2ad69-d117-4dcf-8524-91a6ee99a590 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.841482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.842776 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.843073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.843355 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.844064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.846310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.870016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1363/2744] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:37:38 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:38.873744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.885937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0d4ae-0a96-4e77-9875-b536cba68c35 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.890841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.891584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.891785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.892096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.893860 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.895486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.901715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:38.907883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1382/2745] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:37:38.912507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.923162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643044b-95c6-4f72-8863-8c844966e7e4 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.928373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9691a7-568a-48c7-adca-1aa3adec9da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.929099 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9691a7-568a-48c7-adca-1aa3adec9da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.929388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9691a7-568a-48c7-adca-1aa3adec9da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.929682 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9691a7-568a-48c7-adca-1aa3adec9da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.930404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9691a7-568a-48c7-adca-1aa3adec9da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.932027 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9691a7-568a-48c7-adca-1aa3adec9da5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:38.944549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1364/2746] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:37:38.948863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:38.960808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058219e-cd46-4e82-a545-5bb7267069e0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:38.966019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:38.966921 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:38.967216 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:38.967490 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] PATH_INFO: `/v3/projects/f3f0c2dd35fa4890a64fcc7dcaddc633/users/fc7827c5d9294b8c90f26deff983b8e3/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:38.987170 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f0c2dd35fa4890a64fcc7dcaddc633, user_id=fc7827c5d9294b8c90f26deff983b8e3, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:38.990971 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:39.008766 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1383/2747] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:38 2022] PUT /identity/v3/projects/f3f0c2dd35fa4890a64fcc7dcaddc633/users/fc7827c5d9294b8c90f26deff983b8e3/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:39.014669 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7075d98-2e02-4431-bc31-b4019d7447d2 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:39.015036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7075d98-2e02-4431-bc31-b4019d7447d2 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:39.015394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7075d98-2e02-4431-bc31-b4019d7447d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:39.038852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-f7075d98-2e02-4431-bc31-b4019d7447d2 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:39.040783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f7075d98-2e02-4431-bc31-b4019d7447d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:39.116038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1365/2748] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:39.125649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:39.128746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:39.189040 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc90fa5-96ec-4ca3-a5ed-df04b6f36aeb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:39.192430 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:39.193130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:39.193447 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:39.193738 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:39.203208 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:39.205199 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:39.214883 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1384/2749] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:44.090296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574d16-c314-4413-b590-d698de4c17ce None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.090781 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574d16-c314-4413-b590-d698de4c17ce None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.090879 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574d16-c314-4413-b590-d698de4c17ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.122166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-af574d16-c314-4413-b590-d698de4c17ce None None] MFA Rules not processed for user `fc7827c5d9294b8c90f26deff983b8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:44.123403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-af574d16-c314-4413-b590-d698de4c17ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:44.143816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:44.199401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1366/2750] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:44.204289 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:44.215305 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe575ac4-6442-426d-9fd3-16ad183d40e0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:44.220761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:44.221895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.222194 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.222480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.223191 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:44.225137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:44.256784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1385/2751] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:37:44 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:37:44.258129 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574d16-c314-4413-b590-d698de4c17ce None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:44.270517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574d16-c314-4413-b590-d698de4c17ce None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:44.334185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:44.335053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.335341 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.335621 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.336340 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:44.338339 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:44.358494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1367/2752] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:37:44 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:37:44.362711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:44.373930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdce181-04cf-485b-b495-ddea57161ff5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:44.378685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:44.379338 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.379627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.379911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.380642 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:44.382276 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:44.393465 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1386/2753] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:44.397716 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:44.408436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b78851-45b8-48f0-9303-4fe4b569a191 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:44.413143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:44.414285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.414565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.414978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] PATH_INFO: `/v3/projects/cd91da4bd20948c8a4dd918ac4f88390/users/97b9d773a0c64dc6a6c5284cf73852c3/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.430220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd91da4bd20948c8a4dd918ac4f88390, user_id=97b9d773a0c64dc6a6c5284cf73852c3, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:44.433137 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:44.445968 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1368/2754] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:44 2022] PUT /identity/v3/projects/cd91da4bd20948c8a4dd918ac4f88390/users/97b9d773a0c64dc6a6c5284cf73852c3/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:44.449931 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:44.461405 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05937327-03d1-471d-8f45-563d0d6f2d81 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:44.516019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15508551-5698-460e-9607-bc3cf29fad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:44.516711 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15508551-5698-460e-9607-bc3cf29fad6b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.517535 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15508551-5698-460e-9607-bc3cf29fad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.517813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15508551-5698-460e-9607-bc3cf29fad6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.518700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15508551-5698-460e-9607-bc3cf29fad6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:44.520380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15508551-5698-460e-9607-bc3cf29fad6b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:44.530628 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1387/2755] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:37:44.535719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:44.546442 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b64b7-3c2b-49a2-9992-ecc29c100444 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:44.551320 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:44.552194 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:44.552481 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:44.552771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] PATH_INFO: `/v3/projects/cd91da4bd20948c8a4dd918ac4f88390/users/97b9d773a0c64dc6a6c5284cf73852c3/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:44.554874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd91da4bd20948c8a4dd918ac4f88390, user_id=97b9d773a0c64dc6a6c5284cf73852c3, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:44.557568 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:44.568977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1369/2756] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:37:44 2022] PUT /identity/v3/projects/cd91da4bd20948c8a4dd918ac4f88390/users/97b9d773a0c64dc6a6c5284cf73852c3/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:37:49.226988 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a959c28-594a-4ff2-9666-0b40a54a3303 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:49.227572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a959c28-594a-4ff2-9666-0b40a54a3303 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:49.227572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a959c28-594a-4ff2-9666-0b40a54a3303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:49.259725 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5a959c28-594a-4ff2-9666-0b40a54a3303 None None] MFA Rules not processed for user `97b9d773a0c64dc6a6c5284cf73852c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:49.262595 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5a959c28-594a-4ff2-9666-0b40a54a3303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:49.301356 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:49.355637 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1388/2757] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:49.373684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:49.375657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:49.478816 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d828d-1b99-47ef-bde7-46cb1088ae32 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:49.484331 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:49.485159 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:49.485290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:49.486582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:49.492861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:49.548464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:49.550127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:49.559536 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1370/2758] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:50.635710 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:50.636621 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:50.637164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:50.656358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] MFA Rules not processed for user `fc7827c5d9294b8c90f26deff983b8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:37:50.658343 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:50.665881 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1389/2759] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:37:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:37:56.097302 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:56.099896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:56.168581 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907e632-c045-40c5-9889-1b7a117edbc0 service nova] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:56.172264 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:56.173667 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:56.173907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:56.174205 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:56.176185 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:56.191126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:56.192588 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:56.204103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1371/2760] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:37:57.395951 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:37:57.405481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfa6cc-1bdb-4711-916f-105fb98d6d1a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:37:57.409357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:37:57.410014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:37:57.410330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:37:57.410604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:37:57.412444 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:37:57.427836 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:37:57.429312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:37:57.444323 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1390/2761] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:37:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:38:06.199016 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=50 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:07.435558 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=24 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:10.995858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:11.005059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f728add-7376-4937-bc3f-b51d14a6528d service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:11.009144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:11.009907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:11.010292 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:11.010661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:11.013133 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:11.090520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:11.092003 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:11.104238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1372/2762] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:18.014570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:18.016158 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:18.031365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b56683-36d4-45fa-bc4f-df763cc78b55 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:18.036591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:18.037487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:18.037767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:18.038037 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] PATH_INFO: `/v3/users/493d126879f44b26951920235e7f7730` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:18.039478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493d126879f44b26951920235e7f7730)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:18.041415 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:18.086873 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493d126879f44b26951920235e7f7730'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.095527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493d126879f44b26951920235e7f7730'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.109800 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493d126879f44b26951920235e7f7730'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.110652 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493d126879f44b26951920235e7f7730'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.118473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493d126879f44b26951920235e7f7730'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.124690 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1391/2763] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:38:18 2022] DELETE /identity/v3/users/493d126879f44b26951920235e7f7730 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:38:18.305461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:18.316524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5bb7f-baae-49b1-8af0-58b334090f63 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:18.375254 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:18.376122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:18.376409 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:18.376704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] PATH_INFO: `/v3/projects/3f1255754afb4cd384a11c87a9d717df` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:18.377909 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1255754afb4cd384a11c87a9d717df)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:18.380553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:18.428416 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1255754afb4cd384a11c87a9d717df', 'request_id': 'req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.433929 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1255754afb4cd384a11c87a9d717df', 'request_id': 'req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.441399 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1255754afb4cd384a11c87a9d717df', 'request_id': 'req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.453093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] The token cache is being invalidate because project 3f1255754afb4cd384a11c87a9d717df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:38:18.453388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.454034 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.455589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1373/2764] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:38:18 2022] DELETE /identity/v3/projects/3f1255754afb4cd384a11c87a9d717df => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:38:18.460610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:18.462298 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:18.476048 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27d19e-b26d-4497-a6ea-c677b9337048 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:18.539776 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:18.540673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:18.540990 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:18.541296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] PATH_INFO: `/v3/users/fb82ee83de5d44b6b979dfebd04ccd29` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:18.542539 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb82ee83de5d44b6b979dfebd04ccd29)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:18.544554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:18.595130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb82ee83de5d44b6b979dfebd04ccd29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.602463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb82ee83de5d44b6b979dfebd04ccd29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.615723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb82ee83de5d44b6b979dfebd04ccd29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.616455 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb82ee83de5d44b6b979dfebd04ccd29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.618236 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:18.625476 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb82ee83de5d44b6b979dfebd04ccd29'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.634069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1392/2765] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:38:18 2022] DELETE /identity/v3/users/fb82ee83de5d44b6b979dfebd04ccd29 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:38:18.828307 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:18.829873 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:18.844170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63fbd9-f3a9-4f80-9296-f5a8dcc97538 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:18.866271 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:18.905257 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:18.906146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:18.906453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:18.906727 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] PATH_INFO: `/v3/projects/350f45adac63449687f2f244dc3bf3ce` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:18.907891 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=350f45adac63449687f2f244dc3bf3ce)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:18.909840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:18.957677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350f45adac63449687f2f244dc3bf3ce', 'request_id': 'req-9ac86ea5-8b59-4f47-88a5-6062def04ffa'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.963464 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350f45adac63449687f2f244dc3bf3ce', 'request_id': 'req-9ac86ea5-8b59-4f47-88a5-6062def04ffa'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.970694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350f45adac63449687f2f244dc3bf3ce', 'request_id': 'req-9ac86ea5-8b59-4f47-88a5-6062def04ffa'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.983101 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] The token cache is being invalidate because project 350f45adac63449687f2f244dc3bf3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:38:18.983391 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.984022 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-9ac86ea5-8b59-4f47-88a5-6062def04ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:38:18.985601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1374/2766] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:38:18 2022] DELETE /identity/v3/projects/350f45adac63449687f2f244dc3bf3ce => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:38:25.889134 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:25.890593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:25.950614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3670a-e21f-44f7-88cd-5b50a0943f9e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:25.999892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:26.000538 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:26.000826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:26.001145 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:26.002993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:26.073412 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:26.074778 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:26.085715 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1393/2767] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:28.979346 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:36.080316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=72 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:40.048038 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:40.048590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:40.048590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:40.068369 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] MFA Rules not processed for user `90dedb6ffa9f40fe905f60b023aeb92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:38:40.069623 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:40.130897 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1375/2768] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:38:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:38:40.461782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:40.463329 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:40.475111 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634f2e-45e8-4bfb-b563-eadba119c91c service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:40.478691 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:40.479326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:40.479604 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:40.479878 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:40.481668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:40.493453 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:40.494999 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:40.504013 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1394/2769] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:40.835148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:40.836642 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:40.915330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf34e4-f590-4aa5-aaa0-fa218993f6ad None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:40.920388 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:40.921088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:40.921370 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:40.921645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:40.923412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:40.936091 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:40.937730 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:40.949058 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1376/2770] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:42.936896 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:42.938365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:42.950132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6e40c-a725-4a15-af4a-04c0b6dfcce9 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:42.954673 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:42.955342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:42.955734 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:42.956181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:42.959245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:42.970649 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:42.972099 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:42.989845 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1395/2771] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:50.944434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=57 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:52.553661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:52.555314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:52.636352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a88c17-f9ab-4f5e-a452-dff9891c6df6 service placement] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:52.643898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:52.643898 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:52.644270 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:52.644526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:52.646436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:52.662400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:52.664377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2de394-2bad-422f-907c-f91a55ab9d47 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:52.681019 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1377/2772] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:52.978242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=12 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:39:02.671572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=34 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:39:50.079406 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:39:50.081096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:39:50.098488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909660-0ebc-4e4b-839f-a96c2f68e7ba service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:39:50.104752 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:39:50.105684 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:39:50.105979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:39:50.106413 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:39:50.108318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:39:50.122897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:39:50.124588 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:39:50.141360 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1396/2773] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:39:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:39:53.316417 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:39:53.317010 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:39:53.317138 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:39:53.335507 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] MFA Rules not processed for user `90dedb6ffa9f40fe905f60b023aeb92e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:39:53.336715 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:39:53.343596 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1378/2774] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:39:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:39:53.662431 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:39:53.672408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04706baa-48ec-4d80-b47c-90d57e7ffb0c service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:39:53.675967 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:39:53.676609 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:39:53.677533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:39:53.677811 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:39:53.679658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:39:53.689789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:39:53.691226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85656-4883-4a6e-a1d6-06adf7ecb86b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:39:53.700016 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1397/2775] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:39:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:40:01.125518 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:40:01.141404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3d5ce-8737-40d2-93de-27896c2b8217 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:40:01.148150 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:40:01.148904 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:01.149276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:01.149626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:01.151511 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:40:01.167153 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:40:01.170323 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a820b-a30f-4e0f-8612-b4b260c8b604 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:40:01.184084 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1379/2776] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:40:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:40:03.696277 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=12 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:40:11.178218 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=9 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:40:41.058502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9689f0-4064-4c58-bf20-59179f49313e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:41.059139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9689f0-4064-4c58-bf20-59179f49313e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:41.059139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9689f0-4064-4c58-bf20-59179f49313e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:41.060460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1398/2777] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:40:41.067234 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a83050-24c9-4a76-ac91-fc3dfa7a51ac None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:41.067749 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a83050-24c9-4a76-ac91-fc3dfa7a51ac None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:41.067863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a83050-24c9-4a76-ac91-fc3dfa7a51ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:41.108505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-91a83050-24c9-4a76-ac91-fc3dfa7a51ac None None] MFA Rules not processed for user `3e16aed371fa4c9da4ea02563fed64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:40:41.109784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-91a83050-24c9-4a76-ac91-fc3dfa7a51ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:40:41.117862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1380/2778] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:40:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:40:41.130665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9689f0-4064-4c58-bf20-59179f49313e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:40:41.143190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9689f0-4064-4c58-bf20-59179f49313e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:40:41.147661 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623813a5-7d6c-494e-80cd-535b5aa0f711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:40:41.148731 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623813a5-7d6c-494e-80cd-535b5aa0f711 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:41.149152 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623813a5-7d6c-494e-80cd-535b5aa0f711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:41.149318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623813a5-7d6c-494e-80cd-535b5aa0f711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:41.157004 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623813a5-7d6c-494e-80cd-535b5aa0f711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:40:41.158436 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623813a5-7d6c-494e-80cd-535b5aa0f711 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:40:41.167828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1399/2779] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:40:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:40:41.261251 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb3e7-1993-439b-aa4d-78f91548feff None None] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:41.261681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb3e7-1993-439b-aa4d-78f91548feff None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:41.262044 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fb3e7-1993-439b-aa4d-78f91548feff None None] PATH_INFO: `/` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:41.263733 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1381/2780] 10.222.0.79 () {54 vars in 921 bytes} [Wed Jul 20 19:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:40:41.269592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:41.269592 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:41.269862 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:41.308348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] MFA Rules not processed for user `9918e21a598f4668b5fde7b173b647ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:40:41.309608 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:40:41.319542 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1400/2781] 10.222.0.79 () {58 vars in 1018 bytes} [Wed Jul 20 19:40:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:40:41.330397 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb3e7-1993-439b-aa4d-78f91548feff None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:40:41.341582 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fb3e7-1993-439b-aa4d-78f91548feff None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:40:41.348088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:40:41.348838 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:41.349626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:41.349925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:41.360238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:40:41.361696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:40:41.373053 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1382/2782] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:40:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:40:51.305930 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=15 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:40:51.368418 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=15 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:06.714327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:06.715933 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:06.781078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b159987-0ac5-49a1-b3fe-cb78126b58ac None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:06.785753 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:06.786656 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:06.786939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:06.787217 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] PATH_INFO: `/v3/users/90dedb6ffa9f40fe905f60b023aeb92e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:06.788480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90dedb6ffa9f40fe905f60b023aeb92e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:06.791296 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:06.836373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90dedb6ffa9f40fe905f60b023aeb92e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:06.843767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90dedb6ffa9f40fe905f60b023aeb92e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:06.856050 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90dedb6ffa9f40fe905f60b023aeb92e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:06.856735 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90dedb6ffa9f40fe905f60b023aeb92e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:06.863995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ab42caa-46fb-4d78-92e3-957d1c8548c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90dedb6ffa9f40fe905f60b023aeb92e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:06.870602 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1401/2783] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:06 2022] DELETE /identity/v3/users/90dedb6ffa9f40fe905f60b023aeb92e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:07.042791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.053541 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9bb55-6e28-47dd-9e92-d7abc899b858 service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.115578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.116365 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.116841 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.117322 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] PATH_INFO: `/v3/projects/a1bfa2d4b4084aa294fbd546ebc0f8c5` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.118521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1bfa2d4b4084aa294fbd546ebc0f8c5)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.120433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.169639 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1bfa2d4b4084aa294fbd546ebc0f8c5', 'request_id': 'req-6b0b2680-5d64-4252-bb14-1b4c8550105f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:07.175246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1bfa2d4b4084aa294fbd546ebc0f8c5', 'request_id': 'req-6b0b2680-5d64-4252-bb14-1b4c8550105f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:07.183558 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1bfa2d4b4084aa294fbd546ebc0f8c5', 'request_id': 'req-6b0b2680-5d64-4252-bb14-1b4c8550105f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:07.196545 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] The token cache is being invalidate because project a1bfa2d4b4084aa294fbd546ebc0f8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:07.196828 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:07.197515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6b0b2680-5d64-4252-bb14-1b4c8550105f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:07.199100 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1383/2784] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:07 2022] DELETE /identity/v3/projects/a1bfa2d4b4084aa294fbd546ebc0f8c5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:41:07.204348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.204641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.204940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.227481 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:07.228712 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:07.293739 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1402/2785] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:07.301799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.302096 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.302384 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.322914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:07.324357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:07.333144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1384/2786] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:07.338659 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.349139 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d855e-ddc0-466b-bc64-ca4c1af00473 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.356424 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.357433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.357719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.358020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.358773 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:41:07.359080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.360754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.375335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1403/2787] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:41:07.380612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.393069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe554e4-62ec-4a08-9b3a-03b40601c146 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.399258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.400324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.400662 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.400931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.401636 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.403707 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.441245 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1385/2788] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:07 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:07.445502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.457597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4bbcc-697a-4f28-8356-91262288b2fe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.462036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:07.532147 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.532909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.533213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.533488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.534206 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.536317 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.562315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1404/2789] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:07 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:41:07.566162 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.579161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f164d3-a9af-49b2-86bc-0c1f1f8a35bd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.585368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.586278 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.586570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.586840 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.587578 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.589294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.602282 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1386/2790] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:07.606342 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.617870 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b98ed-9b18-4c98-b726-a0cd1bc9c10c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.625934 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e83a6-07bf-4896-b475-b39c0fca30fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.626644 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e83a6-07bf-4896-b475-b39c0fca30fc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.627038 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e83a6-07bf-4896-b475-b39c0fca30fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.627203 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e83a6-07bf-4896-b475-b39c0fca30fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.627955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e83a6-07bf-4896-b475-b39c0fca30fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.629699 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e83a6-07bf-4896-b475-b39c0fca30fc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.643205 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1405/2791] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:07.649363 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.660478 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c14bb8-37e4-4328-8813-503a706b1e1e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.665962 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.666867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.667148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.667428 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] PATH_INFO: `/v3/projects/23d533381be34e12a89706bbed8f6f92/users/3a8e26ee0fac413c9ca67ff56b6a6eb7/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.701213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d533381be34e12a89706bbed8f6f92, user_id=3a8e26ee0fac413c9ca67ff56b6a6eb7, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:07.705368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:07.724552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1387/2792] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:07 2022] PUT /identity/v3/projects/23d533381be34e12a89706bbed8f6f92/users/3a8e26ee0fac413c9ca67ff56b6a6eb7/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:07.736393 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.737097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.737097 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:07.776660 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] MFA Rules not processed for user `3a8e26ee0fac413c9ca67ff56b6a6eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:07.778297 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:07.881122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1406/2793] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:07.891653 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:07.893900 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:07.931290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:07.984626 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975e924-53d5-44c8-81d3-76065193074e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:07.990857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de0a1a-3239-4887-95fe-80a286536343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9918e21a598f4668b5fde7b173b647ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:07.991601 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de0a1a-3239-4887-95fe-80a286536343 service nova] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:07.991865 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de0a1a-3239-4887-95fe-80a286536343 service nova] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:07.992203 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de0a1a-3239-4887-95fe-80a286536343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:08.001191 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de0a1a-3239-4887-95fe-80a286536343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:08.002817 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de0a1a-3239-4887-95fe-80a286536343 service nova] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:08.013994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1388/2794] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:10.706397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:10.768049 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413d95-4e24-48c5-abd6-b247afc78324 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:10.773385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:10.774312 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:10.774586 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:10.774855 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] PATH_INFO: `/v3/users/3a8e26ee0fac413c9ca67ff56b6a6eb7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:10.776059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8e26ee0fac413c9ca67ff56b6a6eb7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:10.778014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:10.794486 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:10.829149 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8e26ee0fac413c9ca67ff56b6a6eb7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:10.837418 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8e26ee0fac413c9ca67ff56b6a6eb7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:10.849417 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8e26ee0fac413c9ca67ff56b6a6eb7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:10.850119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8e26ee0fac413c9ca67ff56b6a6eb7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:10.857876 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8e26ee0fac413c9ca67ff56b6a6eb7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:10.863849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1407/2795] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:10 2022] DELETE /identity/v3/users/3a8e26ee0fac413c9ca67ff56b6a6eb7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:10.868735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:10.869412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:10.869412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:10.886979 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:10.888177 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:10.950313 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1389/2796] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:10.961308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:10.962777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:11.026226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512acdd-5479-439d-ba43-998f990a98e6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.029911 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a28a-1ed1-4740-8cb5-c6345ce86796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.030566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a28a-1ed1-4740-8cb5-c6345ce86796 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.030852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a28a-1ed1-4740-8cb5-c6345ce86796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.031130 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a28a-1ed1-4740-8cb5-c6345ce86796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.040345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a28a-1ed1-4740-8cb5-c6345ce86796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.042082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a28a-1ed1-4740-8cb5-c6345ce86796 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.051751 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1408/2797] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:11.330548 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.332015 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:11.346334 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd8e7c-0a56-4c07-b2e1-e931effbe20e None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.351105 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.351974 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.352252 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.352523 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] PATH_INFO: `/v3/projects/23d533381be34e12a89706bbed8f6f92` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.353764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d533381be34e12a89706bbed8f6f92)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.355759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.404121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d533381be34e12a89706bbed8f6f92', 'request_id': 'req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:11.409719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d533381be34e12a89706bbed8f6f92', 'request_id': 'req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:11.417146 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d533381be34e12a89706bbed8f6f92', 'request_id': 'req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:11.428461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] The token cache is being invalidate because project 23d533381be34e12a89706bbed8f6f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:11.428754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:11.430151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-1f8c5ba4-370d-4d18-a3a8-83e6f11ee142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:11.431753 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1390/2798] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:11 2022] DELETE /identity/v3/projects/23d533381be34e12a89706bbed8f6f92 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:11.436584 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.436874 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.437175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.455212 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:11.456390 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:11.642188 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1409/2799] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:11.650125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.650414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.650695 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.669473 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:11.670694 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:11.678520 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1391/2800] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:11.683410 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.693315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b58c2-8bc7-4fab-ae75-9a434fbbffde None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.698533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.699201 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.699488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.699793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.700530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:41:11.700841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.702457 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.714409 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1410/2801] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:41:11.718587 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.728680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead932f5-66d0-40b3-8107-af2e279197eb None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.733619 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.734274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.734560 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.734837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.735538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.737490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.763256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1392/2802] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:11 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:11.767397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.777545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7c804-57d3-40db-9a8a-6c04ea82e189 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.798058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:11.836380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.837069 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.837373 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.837645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.838327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.840370 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.861164 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1411/2803] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:11 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:11.865337 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.875725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0861b0-3254-4ecd-9dc8-6a3596b2cc15 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.881004 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.881683 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.881976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.882327 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.883166 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.884779 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.886196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:11.897196 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1393/2804] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:11.902110 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.912321 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66479096-9ea8-4850-a8bf-b432239f6dd8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.917777 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde246ba-c994-4bdb-865f-7d4d47c74d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.918435 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde246ba-c994-4bdb-865f-7d4d47c74d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.918720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde246ba-c994-4bdb-865f-7d4d47c74d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.918997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde246ba-c994-4bdb-865f-7d4d47c74d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.919723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde246ba-c994-4bdb-865f-7d4d47c74d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.921357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde246ba-c994-4bdb-865f-7d4d47c74d25 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.931467 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1412/2805] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:11.935471 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:11.945989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327f848-7aff-4b7e-9574-a7eac6734152 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:11.951082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:11.951995 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.952280 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.952559 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] PATH_INFO: `/v3/projects/670ca1de8c824f549356fb7b8638c6f8/users/b7bb0b9d1bd7408eb8877756bc76ba7f/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:11.967934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670ca1de8c824f549356fb7b8638c6f8, user_id=b7bb0b9d1bd7408eb8877756bc76ba7f, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:11.970561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:11.985586 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1394/2806] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:11 2022] PUT /identity/v3/projects/670ca1de8c824f549356fb7b8638c6f8/users/b7bb0b9d1bd7408eb8877756bc76ba7f/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:11.997386 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:11.997682 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:11.997991 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.025184 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] MFA Rules not processed for user `b7bb0b9d1bd7408eb8877756bc76ba7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:12.026420 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:12.090046 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1413/2807] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:12.094648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.158450 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23658d5-8e47-44d5-9412-294089dd4651 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.163115 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.164717 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.164967 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.165250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.166256 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.168161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.193585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1395/2808] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:12 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:12.197385 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.207993 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05e969-8f9f-4fe7-b53e-39e25744462b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.270310 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.270981 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.271273 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.271553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.272254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.275115 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.294936 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1414/2809] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:12 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:41:12.299057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.309330 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb441514-8e34-41ad-889e-8ba5159f89f1 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.313992 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.314645 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.314931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.315204 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.315958 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.317737 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.329063 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1396/2810] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:12.333241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.344066 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9b8d9-ba2c-40de-868b-f40d3a4eed5e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.348827 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.349708 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.349992 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.350270 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] PATH_INFO: `/v3/projects/20c689c0f94b496da3464312b664fc17/users/9438c831984d4aba874f973101993dec/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.364909 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c689c0f94b496da3464312b664fc17, user_id=9438c831984d4aba874f973101993dec, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.367591 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.375779 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:12.382724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1415/2811] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:12 2022] PUT /identity/v3/projects/20c689c0f94b496da3464312b664fc17/users/9438c831984d4aba874f973101993dec/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:12.386585 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.396504 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976aaf1-ed00-41ac-a39c-68b1fcfdc18c admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.452782 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857f963-3541-4d37-9609-68a1a096cc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.453494 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857f963-3541-4d37-9609-68a1a096cc9b admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.453784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857f963-3541-4d37-9609-68a1a096cc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.454067 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857f963-3541-4d37-9609-68a1a096cc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.454797 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857f963-3541-4d37-9609-68a1a096cc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.456404 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857f963-3541-4d37-9609-68a1a096cc9b admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.467047 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1397/2812] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:41:12.471263 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.481784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34958-920f-4ffc-9991-209711657a9d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.486299 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.487167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.487501 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.487719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] PATH_INFO: `/v3/projects/20c689c0f94b496da3464312b664fc17/users/9438c831984d4aba874f973101993dec/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.489810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c689c0f94b496da3464312b664fc17, user_id=9438c831984d4aba874f973101993dec, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.492408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.502744 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1416/2813] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:12 2022] PUT /identity/v3/projects/20c689c0f94b496da3464312b664fc17/users/9438c831984d4aba874f973101993dec/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:12.513526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.513821 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.514106 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.542347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] MFA Rules not processed for user `9438c831984d4aba874f973101993dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:12.543574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:12.574382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:12.613711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1398/2814] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:12.625838 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:12.627554 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:12.685019 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a15e-17c9-4935-8b80-1b17e31a1742 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:12.688635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a412e50-2b11-412f-8479-b42f6cf7b7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:12.689308 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412e50-2b11-412f-8479-b42f6cf7b7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:12.689600 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412e50-2b11-412f-8479-b42f6cf7b7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:12.689879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412e50-2b11-412f-8479-b42f6cf7b7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:12.700854 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a412e50-2b11-412f-8479-b42f6cf7b7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:12.702368 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a412e50-2b11-412f-8479-b42f6cf7b7c1 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:12.712775 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1417/2815] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:13.100565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:13.162591 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47b7b7-8e19-4763-a37f-1624c9e2a7b7 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:13.167836 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:13.168736 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:13.169074 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:13.169379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] PATH_INFO: `/v3/users/b7bb0b9d1bd7408eb8877756bc76ba7f` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:13.170571 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7bb0b9d1bd7408eb8877756bc76ba7f)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:13.172508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:13.218901 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7bb0b9d1bd7408eb8877756bc76ba7f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.225925 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7bb0b9d1bd7408eb8877756bc76ba7f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.238073 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7bb0b9d1bd7408eb8877756bc76ba7f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.238754 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7bb0b9d1bd7408eb8877756bc76ba7f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.247505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7bb0b9d1bd7408eb8877756bc76ba7f'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.253436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1399/2816] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:13 2022] DELETE /identity/v3/users/b7bb0b9d1bd7408eb8877756bc76ba7f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:13.258573 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:13.259084 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:13.259190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:13.276460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:13.277685 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:13.338021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1418/2817] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:13.347517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:13.348934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:13.403094 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:13.409393 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169d198-a34e-4fc6-a042-289bc8b3a081 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:13.414274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:13.414924 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:13.415209 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:13.415735 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:13.424808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:13.426288 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:13.436342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1400/2818] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:13.712702 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:13.714237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:13.726678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631e73f-1060-4edb-bfe8-2c713c8203fe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:13.732336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:13.733327 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:13.733617 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:13.733903 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] PATH_INFO: `/v3/projects/670ca1de8c824f549356fb7b8638c6f8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:13.735095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=670ca1de8c824f549356fb7b8638c6f8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:13.737053 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:13.786309 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670ca1de8c824f549356fb7b8638c6f8', 'request_id': 'req-64c371ba-04b4-4103-b148-da8eeb2069fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.793032 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670ca1de8c824f549356fb7b8638c6f8', 'request_id': 'req-64c371ba-04b4-4103-b148-da8eeb2069fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.800469 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670ca1de8c824f549356fb7b8638c6f8', 'request_id': 'req-64c371ba-04b4-4103-b148-da8eeb2069fb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.812635 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] The token cache is being invalidate because project 670ca1de8c824f549356fb7b8638c6f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:13.812963 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.813587 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.815095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1419/2819] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:13 2022] DELETE /identity/v3/projects/670ca1de8c824f549356fb7b8638c6f8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:13.819087 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:13.828830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b33d27-6fc0-410e-9b1c-d8991a51ca7a service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:13.886092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:13.886966 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:13.887250 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:13.887524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] PATH_INFO: `/v3/users/9438c831984d4aba874f973101993dec` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:13.888739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9438c831984d4aba874f973101993dec)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:13.890668 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:13.937722 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9438c831984d4aba874f973101993dec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.944615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9438c831984d4aba874f973101993dec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.957847 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9438c831984d4aba874f973101993dec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.958549 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9438c831984d4aba874f973101993dec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.973000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-bfad81c7-88dc-4b29-a23d-6a1bd5468ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9438c831984d4aba874f973101993dec'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:13.979977 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1401/2820] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:13 2022] DELETE /identity/v3/users/9438c831984d4aba874f973101993dec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:14.263426 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.264926 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:14.280647 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c371ba-04b4-4103-b148-da8eeb2069fb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.289084 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:14.340175 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.341073 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.341362 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.341643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] PATH_INFO: `/v3/projects/20c689c0f94b496da3464312b664fc17` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.342821 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c689c0f94b496da3464312b664fc17)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.344760 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.391058 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c689c0f94b496da3464312b664fc17', 'request_id': 'req-aacfdb98-16fa-41c7-8c88-3c7e896a0937'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:14.396451 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c689c0f94b496da3464312b664fc17', 'request_id': 'req-aacfdb98-16fa-41c7-8c88-3c7e896a0937'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:14.403810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c689c0f94b496da3464312b664fc17', 'request_id': 'req-aacfdb98-16fa-41c7-8c88-3c7e896a0937'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:14.414834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] The token cache is being invalidate because project 20c689c0f94b496da3464312b664fc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:14.415129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:14.416014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aacfdb98-16fa-41c7-8c88-3c7e896a0937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:14.417577 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1420/2821] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:14 2022] DELETE /identity/v3/projects/20c689c0f94b496da3464312b664fc17 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:14.422634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.423116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.423222 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.441276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:14.442510 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:14.502482 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1402/2822] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:14.511791 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.512078 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.512354 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.531157 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:14.532348 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:14.543034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1421/2823] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:14.547303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.557634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5a892-e23f-4223-9104-4a89e78e419f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.562483 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.563131 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.563413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.563689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.564414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:41:14.564719 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.566368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.568818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:14.580413 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1403/2824] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:41:14.584705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.594144 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf71dce-8e20-40e2-8bd1-44196504cfae None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.598782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.599404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.599692 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.599969 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.600657 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.603306 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.627781 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1422/2825] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:14 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:14.632648 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.642751 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd141c6-9b0a-45ee-acb5-cdf79d698fba admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.700723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.701739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.701739 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.701959 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.702685 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.704610 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.725246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1404/2826] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:14 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:14.729545 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.740029 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeebac8-d512-4d82-9e5d-c6a61984cbb5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.744599 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.745287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.745572 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.745841 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.746561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.748437 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.759650 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1423/2827] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:14.763631 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.774326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e28bd0-00f9-4a7d-9007-ab769fae46dd admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.779030 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd5ad5-9c89-484e-96d9-fb616b6e21a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.779689 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd5ad5-9c89-484e-96d9-fb616b6e21a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.779973 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd5ad5-9c89-484e-96d9-fb616b6e21a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.780276 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd5ad5-9c89-484e-96d9-fb616b6e21a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.781046 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd5ad5-9c89-484e-96d9-fb616b6e21a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.782657 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd5ad5-9c89-484e-96d9-fb616b6e21a3 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.792921 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1405/2828] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:14.797051 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.807077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2943a-07eb-46ff-b90d-ce16e6184d63 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:14.811658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:14.812533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.812813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.813119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] PATH_INFO: `/v3/projects/c5c0a5e6da5649adbf0bc6b12e5d9793/users/4c3313e72bf3410e844fd9d74788a530/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.828000 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c0a5e6da5649adbf0bc6b12e5d9793, user_id=4c3313e72bf3410e844fd9d74788a530, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:14.830683 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:14.843840 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1424/2829] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:14 2022] PUT /identity/v3/projects/c5c0a5e6da5649adbf0bc6b12e5d9793/users/4c3313e72bf3410e844fd9d74788a530/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:14.854272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:14.854565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:14.854842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:14.883604 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] MFA Rules not processed for user `4c3313e72bf3410e844fd9d74788a530`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:14.885348 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:14.949414 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1406/2830] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:14.953879 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:14.997746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:15.021061 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576005d5-fd0a-4797-8f59-5c04302b2049 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.025865 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.026518 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.026798 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.027072 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.027756 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.029831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.056085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1425/2831] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:14 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:15.060253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:15.072093 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4acb-f840-4112-8417-230b90cb77ec None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.127866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.128553 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.128837 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.129152 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.129842 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.131760 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.141064 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:15.154228 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1407/2832] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:15 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:41:15.158155 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:15.169065 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f455a14-66da-4af9-b5a6-ab8d56b5acd5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.173544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.174197 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.174478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.174767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.175504 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.177122 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.188168 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1426/2833] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:15.192476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:15.203392 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527383b-51d0-497b-a62d-528da64091d9 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.208170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.209112 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.209402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.209684 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] PATH_INFO: `/v3/projects/56262d2cc5704055bf12e78f765db7f3/users/4c1f36cbd524424e92bb0870df68a6af/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.224352 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56262d2cc5704055bf12e78f765db7f3, user_id=4c1f36cbd524424e92bb0870df68a6af, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.227696 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.240681 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1408/2834] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:15 2022] PUT /identity/v3/projects/56262d2cc5704055bf12e78f765db7f3/users/4c1f36cbd524424e92bb0870df68a6af/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:41:15.245511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:15.256235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485c2ed-6569-499b-87ab-3b6c0cf9ff2f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.317645 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5302-8b5e-404e-8c5f-8837e60a2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.318397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5302-8b5e-404e-8c5f-8837e60a2d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.318507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5302-8b5e-404e-8c5f-8837e60a2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.318740 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5302-8b5e-404e-8c5f-8837e60a2d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.319523 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5302-8b5e-404e-8c5f-8837e60a2d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.321142 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5302-8b5e-404e-8c5f-8837e60a2d58 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.330932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1427/2835] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:15.335202 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:15.345401 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c3744-14fb-4373-9e22-23f8e296629e admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.350175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.351574 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.351916 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.352220 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] PATH_INFO: `/v3/projects/56262d2cc5704055bf12e78f765db7f3/users/4c1f36cbd524424e92bb0870df68a6af/roles/689534f62373409f9d02ea4acc335ab7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.354238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56262d2cc5704055bf12e78f765db7f3, user_id=4c1f36cbd524424e92bb0870df68a6af, role_id=689534f62373409f9d02ea4acc335ab7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.356835 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.369069 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1409/2836] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:15 2022] PUT /identity/v3/projects/56262d2cc5704055bf12e78f765db7f3/users/4c1f36cbd524424e92bb0870df68a6af/roles/689534f62373409f9d02ea4acc335ab7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:15.379267 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.379565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.379850 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.406678 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] MFA Rules not processed for user `4c1f36cbd524424e92bb0870df68a6af`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:15.407887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:15.475252 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1428/2837] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:15.487020 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:15.488866 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:15.546452 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713b6d0-54f7-43e3-af89-91e075b52c32 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:15.550070 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a5aff-a323-455a-865d-b61ace094ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:15.550711 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a5aff-a323-455a-865d-b61ace094ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:15.550996 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a5aff-a323-455a-865d-b61ace094ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:15.551272 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a5aff-a323-455a-865d-b61ace094ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:15.561028 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a5aff-a323-455a-865d-b61ace094ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:15.562468 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a5aff-a323-455a-865d-b61ace094ddf service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:15.572375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1410/2838] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:15.980121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:16.015585 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:16.046669 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba519c6-b6c7-427e-9702-69df508a05c2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:16.051583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:16.052537 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:16.052818 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:16.053124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] PATH_INFO: `/v3/users/4c3313e72bf3410e844fd9d74788a530` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:16.054613 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3313e72bf3410e844fd9d74788a530)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:16.056565 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:16.104007 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3313e72bf3410e844fd9d74788a530'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.110754 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3313e72bf3410e844fd9d74788a530'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.122530 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3313e72bf3410e844fd9d74788a530'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.123349 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3313e72bf3410e844fd9d74788a530'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.135782 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3313e72bf3410e844fd9d74788a530'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.142132 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1429/2839] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:15 2022] DELETE /identity/v3/users/4c3313e72bf3410e844fd9d74788a530 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:16.146759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:16.147082 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:16.147561 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:16.166989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:16.168212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:16.230849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1411/2840] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:16.240527 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:16.242063 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:16.299119 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96e3f9-5b1e-4dc2-9a6d-9615b7a5b232 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:16.302793 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:16.303434 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:16.303723 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:16.304129 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:16.314020 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:16.315446 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:16.325688 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1430/2841] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:16.601932 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:16.603403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:16.615647 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ae377-19ce-41c2-b582-533312995a90 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:16.621394 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:16.623342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:16.623342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:16.623342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] PATH_INFO: `/v3/projects/c5c0a5e6da5649adbf0bc6b12e5d9793` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:16.625021 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c0a5e6da5649adbf0bc6b12e5d9793)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:16.627059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:16.686018 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c0a5e6da5649adbf0bc6b12e5d9793', 'request_id': 'req-323880c5-fa03-49cb-ab36-6ce0445db162'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.689783 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:16.692140 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c0a5e6da5649adbf0bc6b12e5d9793', 'request_id': 'req-323880c5-fa03-49cb-ab36-6ce0445db162'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.702049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c0a5e6da5649adbf0bc6b12e5d9793', 'request_id': 'req-323880c5-fa03-49cb-ab36-6ce0445db162'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.714618 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] The token cache is being invalidate because project c5c0a5e6da5649adbf0bc6b12e5d9793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:16.714914 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.715552 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.717136 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1412/2842] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:16 2022] DELETE /identity/v3/projects/c5c0a5e6da5649adbf0bc6b12e5d9793 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:16.721137 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:16.731543 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31b49-a5a3-4185-96a1-3b097777bbf3 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:16.783940 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:16.796330 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:16.797266 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:16.797550 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:16.797851 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] PATH_INFO: `/v3/users/4c1f36cbd524424e92bb0870df68a6af` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:16.799135 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1f36cbd524424e92bb0870df68a6af)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:16.801131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:16.848566 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1f36cbd524424e92bb0870df68a6af'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.856280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1f36cbd524424e92bb0870df68a6af'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.869225 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1f36cbd524424e92bb0870df68a6af'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.869922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1f36cbd524424e92bb0870df68a6af'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.877228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1fb8606f-9c24-4df9-9183-5abe76395c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1f36cbd524424e92bb0870df68a6af'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:16.883228 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1431/2843] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:16 2022] DELETE /identity/v3/users/4c1f36cbd524424e92bb0870df68a6af => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:17.175400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.177049 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:17.190386 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323880c5-fa03-49cb-ab36-6ce0445db162 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.252671 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.253614 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.253999 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.254275 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] PATH_INFO: `/v3/projects/56262d2cc5704055bf12e78f765db7f3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.255476 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56262d2cc5704055bf12e78f765db7f3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.257871 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.312036 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56262d2cc5704055bf12e78f765db7f3', 'request_id': 'req-56333df9-d705-42b5-85be-ebd08993e0f4'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:17.319606 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56262d2cc5704055bf12e78f765db7f3', 'request_id': 'req-56333df9-d705-42b5-85be-ebd08993e0f4'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:17.329703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56262d2cc5704055bf12e78f765db7f3', 'request_id': 'req-56333df9-d705-42b5-85be-ebd08993e0f4'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:17.342978 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] The token cache is being invalidate because project 56262d2cc5704055bf12e78f765db7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:17.343266 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:17.343928 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-56333df9-d705-42b5-85be-ebd08993e0f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:17.345508 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1413/2844] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:17 2022] DELETE /identity/v3/projects/56262d2cc5704055bf12e78f765db7f3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:17.350697 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.351789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.351912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.369181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:17.370402 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:17.430852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1432/2845] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:17.490570 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.490858 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.491141 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.509794 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:17.510987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:17.518852 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1414/2846] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:17.523593 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.533379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c458520-d170-4684-8687-159f30cd60d3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.538250 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.538944 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.539187 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.539432 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.540243 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:41:17.540810 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.542463 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.554372 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1433/2847] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:41:17.558905 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.568253 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98a2b4-434b-4906-bb63-0f5b24007ffe None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.574026 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.574710 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.574997 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.575303 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.576102 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.577982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.603790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1415/2848] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:17 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:41:17.608100 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.619643 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372039-7797-4b9e-81dd-2e9cb57623c7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.676937 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.677607 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.677895 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.678173 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.678867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.680768 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.688627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:17.704831 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1434/2849] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:17 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:17.709059 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.720189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308aa513-e716-45ea-804e-647265d24892 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.724838 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.725524 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.725812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.726088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.726818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.728412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.738874 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1416/2850] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:17.743474 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.753521 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec7d60-21c8-432f-ae04-99afd9ab53ca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.759076 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c1847-2929-4bfe-bbf5-05f3b86425ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.759747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c1847-2929-4bfe-bbf5-05f3b86425ae admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.760036 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c1847-2929-4bfe-bbf5-05f3b86425ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.760315 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c1847-2929-4bfe-bbf5-05f3b86425ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.761085 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c1847-2929-4bfe-bbf5-05f3b86425ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.762664 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c1847-2929-4bfe-bbf5-05f3b86425ae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.773551 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1435/2851] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:17.777780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.787766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf5b9-3a77-477c-9ee2-f2aa8e57d55a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:17.792629 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:17.793537 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.793814 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.794085 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] PATH_INFO: `/v3/projects/1329ec7cb8f24270a7c52f219836b7b7/users/72012737ba2e47fdbd9d5ec39ec6585d/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.808375 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1329ec7cb8f24270a7c52f219836b7b7, user_id=72012737ba2e47fdbd9d5ec39ec6585d, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:17.811151 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:17.824702 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1417/2852] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:17 2022] PUT /identity/v3/projects/1329ec7cb8f24270a7c52f219836b7b7/users/72012737ba2e47fdbd9d5ec39ec6585d/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:17.834790 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:17.835156 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:17.836096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:17.862814 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] MFA Rules not processed for user `72012737ba2e47fdbd9d5ec39ec6585d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:17.864022 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:17.927378 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1436/2853] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:17.939590 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:17.941310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:17.953273 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:18.001725 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abe479-97d1-48b9-8a2e-773557d8aa7d admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:18.005433 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6417c-6515-4069-9380-2866eb75f669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:18.006081 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6417c-6515-4069-9380-2866eb75f669 service cinder] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:18.006368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6417c-6515-4069-9380-2866eb75f669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:18.006644 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6417c-6515-4069-9380-2866eb75f669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:18.016546 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6417c-6515-4069-9380-2866eb75f669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:18.018031 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6417c-6515-4069-9380-2866eb75f669 service cinder] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:18.031324 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1418/2854] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:27.924619 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:28.022833 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=22 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:35.510178 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:35.579439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac87ea-2290-4385-ab09-3b5657ab9be6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:35.583955 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:35.584826 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:35.585358 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:35.585842 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] PATH_INFO: `/v3/users/72012737ba2e47fdbd9d5ec39ec6585d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:35.587077 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72012737ba2e47fdbd9d5ec39ec6585d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:35.589027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:35.634828 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72012737ba2e47fdbd9d5ec39ec6585d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:35.641709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72012737ba2e47fdbd9d5ec39ec6585d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:35.653326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72012737ba2e47fdbd9d5ec39ec6585d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:35.653995 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72012737ba2e47fdbd9d5ec39ec6585d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:35.661224 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72012737ba2e47fdbd9d5ec39ec6585d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:35.667906 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1437/2855] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:41:35 2022] DELETE /identity/v3/users/72012737ba2e47fdbd9d5ec39ec6585d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:35.672784 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128870a1-585d-44d8-a614-0df75319223f None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:35.673314 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128870a1-585d-44d8-a614-0df75319223f None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:35.673444 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128870a1-585d-44d8-a614-0df75319223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:35.691080 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-128870a1-585d-44d8-a614-0df75319223f None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:35.692296 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-128870a1-585d-44d8-a614-0df75319223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:35.751818 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1419/2856] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:35.762248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:35.763922 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:35.821941 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259caa58-5b96-4ad4-be4a-a6553fc25a98 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:35.825665 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6afb0-3ce7-4d95-8298-2a949dc3161b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:35.826319 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6afb0-3ce7-4d95-8298-2a949dc3161b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:35.826610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6afb0-3ce7-4d95-8298-2a949dc3161b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:35.826892 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6afb0-3ce7-4d95-8298-2a949dc3161b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:35.836637 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6afb0-3ce7-4d95-8298-2a949dc3161b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:35.838136 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6afb0-3ce7-4d95-8298-2a949dc3161b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:35.847820 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1438/2857] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:36.131309 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128870a1-585d-44d8-a614-0df75319223f None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.133635 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-128870a1-585d-44d8-a614-0df75319223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:36.147479 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128870a1-585d-44d8-a614-0df75319223f None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.152170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.153882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.154178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.154459 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] PATH_INFO: `/v3/projects/1329ec7cb8f24270a7c52f219836b7b7` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.155661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1329ec7cb8f24270a7c52f219836b7b7)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.157634 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.206515 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1329ec7cb8f24270a7c52f219836b7b7', 'request_id': 'req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:36.212075 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1329ec7cb8f24270a7c52f219836b7b7', 'request_id': 'req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:36.218846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1329ec7cb8f24270a7c52f219836b7b7', 'request_id': 'req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:36.231092 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] The token cache is being invalidate because project 1329ec7cb8f24270a7c52f219836b7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:41:36.231382 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:36.232013 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-f59bbb6b-c9cd-4a42-aa7a-2d74b51a45ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:41:36.233566 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1420/2858] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:41:36 2022] DELETE /identity/v3/projects/1329ec7cb8f24270a7c52f219836b7b7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:36.238472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.238765 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.239096 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.257275 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:36.258473 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:36.275641 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:36.321240 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1439/2859] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:36.329708 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e806f18-ced1-4014-812f-2870289703aa None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.330029 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e806f18-ced1-4014-812f-2870289703aa None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.330328 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e806f18-ced1-4014-812f-2870289703aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.349109 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-8e806f18-ced1-4014-812f-2870289703aa None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:36.350310 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8e806f18-ced1-4014-812f-2870289703aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:36.358687 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1421/2860] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:36.362867 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.373369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1db08-8515-4350-87c0-4a5844ded4d5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.377849 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.378498 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.378784 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.379060 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.379785 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:41:36.380095 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.381767 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.393487 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1440/2861] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:41:36.397743 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e806f18-ced1-4014-812f-2870289703aa None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.407116 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e806f18-ced1-4014-812f-2870289703aa None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.411759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.412412 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.412699 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.413528 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.414260 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.416440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.441926 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1422/2862] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:41:36 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:36.446610 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.456511 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60bdd8-ff06-4d42-bc6f-d95de91a6850 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.515357 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.516052 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.516239 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.516517 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.517251 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.519167 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.539772 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1441/2863] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:41:36 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:41:36.544982 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.555942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e5be5-21cb-4815-add8-e548e8dc3e01 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.560539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.561239 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.561526 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.561812 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.562539 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.564174 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.575603 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1423/2864] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:41:36.579575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.590014 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0998f-dfe2-4e0a-a432-596860b25b8f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.594718 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43b378-942c-4377-8d34-5cab8d5021a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.595403 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b378-942c-4377-8d34-5cab8d5021a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.595689 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b378-942c-4377-8d34-5cab8d5021a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.595989 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b378-942c-4377-8d34-5cab8d5021a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.596714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43b378-942c-4377-8d34-5cab8d5021a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.598352 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43b378-942c-4377-8d34-5cab8d5021a8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.608369 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1442/2865] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:41:36.612349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.622846 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748a305-dec0-445e-9946-ea2d461f2ce6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.627491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.628373 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.628660 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.628976 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] PATH_INFO: `/v3/projects/d126312830ce4625a6a74077ae072420/users/dcdaabfe604145ed98fd93c4521d6829/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.644529 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d126312830ce4625a6a74077ae072420, user_id=dcdaabfe604145ed98fd93c4521d6829, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.647207 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.660000 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1424/2866] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:41:36 2022] PUT /identity/v3/projects/d126312830ce4625a6a74077ae072420/users/dcdaabfe604145ed98fd93c4521d6829/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:41:36.670579 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.670912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.671210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.700449 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] MFA Rules not processed for user `dcdaabfe604145ed98fd93c4521d6829`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:41:36.701918 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:36.766388 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1443/2867] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:41:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:41:36.778672 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:36.780521 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:36.805517 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:36.863823 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7ede9-3fa9-47a3-992e-1ac93d6560f8 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:36.868703 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4e27030ec042bca0c7d301998e018e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:36.869376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:36.869665 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:36.869944 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:36.878400 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:36.879838 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:36.890436 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1425/2868] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:46.763569 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:46.886294 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=26 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:55.672717 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:55.674202 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:55.743605 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab0065-6f58-4f6d-8225-0bd8714b7a55 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:55.748783 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:55.749462 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:55.749746 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:55.750021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:55.751794 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:55.763564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:55.765163 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2a0d5-c861-4e87-993b-5b5bcd5843d2 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:55.775047 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1444/2869] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:05.770737 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=32 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:08.001849 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:08.067831 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622b29f-7f91-4aca-9a83-fb2076c01ee9 service glance] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:08.072612 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:08.073589 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.073994 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.074338 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] PATH_INFO: `/v3/users/dcdaabfe604145ed98fd93c4521d6829` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.075597 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcdaabfe604145ed98fd93c4521d6829)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:08.077774 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:08.127765 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcdaabfe604145ed98fd93c4521d6829'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.134987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcdaabfe604145ed98fd93c4521d6829'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.147630 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcdaabfe604145ed98fd93c4521d6829'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.148367 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcdaabfe604145ed98fd93c4521d6829'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.157808 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcdaabfe604145ed98fd93c4521d6829'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.164088 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1426/2870] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:42:08 2022] DELETE /identity/v3/users/dcdaabfe604145ed98fd93c4521d6829 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:08.169627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.170120 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.170288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.187974 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:08.189242 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:08.251316 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1445/2871] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:08.261357 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:08.262991 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:08.324723 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992350b4-47d4-424b-8228-288893a9c1e6 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:08.328377 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85886f-2a16-4687-9c3b-0e32817aa093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:08.329062 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85886f-2a16-4687-9c3b-0e32817aa093 service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.329347 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85886f-2a16-4687-9c3b-0e32817aa093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.329627 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85886f-2a16-4687-9c3b-0e32817aa093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.338565 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85886f-2a16-4687-9c3b-0e32817aa093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:08.340763 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85886f-2a16-4687-9c3b-0e32817aa093 service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:08.350144 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1427/2872] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:08.632958 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:08.634379 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:08.650597 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc1504-74fe-4485-8184-3c29f06096ee None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:08.656288 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:08.657207 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.657626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.657897 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] PATH_INFO: `/v3/projects/d126312830ce4625a6a74077ae072420` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.659249 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d126312830ce4625a6a74077ae072420)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:08.661292 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:08.710268 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd126312830ce4625a6a74077ae072420', 'request_id': 'req-15721f42-f69c-4d34-8539-e2a942925bc1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.716761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd126312830ce4625a6a74077ae072420', 'request_id': 'req-15721f42-f69c-4d34-8539-e2a942925bc1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.724274 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd126312830ce4625a6a74077ae072420', 'request_id': 'req-15721f42-f69c-4d34-8539-e2a942925bc1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.735339 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] The token cache is being invalidate because project d126312830ce4625a6a74077ae072420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:42:08.735626 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.736241 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-15721f42-f69c-4d34-8539-e2a942925bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:08.737808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1446/2873] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:42:08 2022] DELETE /identity/v3/projects/d126312830ce4625a6a74077ae072420 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:08.742780 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.743071 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.743351 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.761126 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:08.762861 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:08.779980 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:08.824583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1428/2874] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:08.832336 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.832625 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.832968 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.851121 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:08.852326 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:08.860480 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1447/2875] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:08.864790 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:08.875490 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eb1eb-fcfa-4f89-a694-2fae9249ec48 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:08.880143 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:08.880791 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.881103 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.881379 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.882098 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80250) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:42:08.882403 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:08.883989 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:08.896227 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1429/2876] 10.222.0.79 () {58 vars in 1152 bytes} [Wed Jul 20 19:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:42:08.900226 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:08.909887 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab6efe-c935-4b2b-82a9-eb897110f235 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:08.914636 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:08.915285 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:08.915575 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:08.915852 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:08.916808 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:08.918932 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:08.944458 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1448/2877] 10.222.0.79 () {60 vars in 1152 bytes} [Wed Jul 20 19:42:08 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:42:08.948580 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:08.959469 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0da206-c2d2-4c72-852a-a82bd89f68b2 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:09.018376 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:09.019122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:09.019453 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:09.019768 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:09.020600 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:09.022764 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:09.042896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1430/2878] 10.222.0.79 () {60 vars in 1143 bytes} [Wed Jul 20 19:42:08 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:42:09.047131 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:09.057389 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99de193-3a23-457d-8de9-6d7102e57a6b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:09.062470 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:09.063148 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:09.063428 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:09.063700 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:09.064429 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:09.066059 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:09.076980 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1449/2879] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:42:09.080893 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:09.091402 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b671-fb75-4d0e-b3e1-753cac5987f5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:09.096611 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eccbe-292c-48d1-afed-dd230b97af6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:09.097325 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eccbe-292c-48d1-afed-dd230b97af6e admin admin] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:09.097615 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eccbe-292c-48d1-afed-dd230b97af6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:09.097896 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eccbe-292c-48d1-afed-dd230b97af6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:09.098632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eccbe-292c-48d1-afed-dd230b97af6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:09.100274 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eccbe-292c-48d1-afed-dd230b97af6e admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:09.111189 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1431/2880] 10.222.0.79 () {58 vars in 1121 bytes} [Wed Jul 20 19:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:42:09.115813 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:09.126324 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783ccd-e57f-4f3b-b2ca-4ee7381eb7e5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:09.130861 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:09.131733 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:09.132017 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:09.132295 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] PATH_INFO: `/v3/projects/67f87e7aec9f4fa7a23ba9ea65d2d733/users/f9038a6d909342fb979b2a42534ff93e/roles/cceb4e5cab5449d893b958c6130869e3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:09.147260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f87e7aec9f4fa7a23ba9ea65d2d733, user_id=f9038a6d909342fb979b2a42534ff93e, role_id=cceb4e5cab5449d893b958c6130869e3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:09.150045 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:09.163234 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1450/2881] 10.222.0.79 () {60 vars in 1482 bytes} [Wed Jul 20 19:42:09 2022] PUT /identity/v3/projects/67f87e7aec9f4fa7a23ba9ea65d2d733/users/f9038a6d909342fb979b2a42534ff93e/roles/cceb4e5cab5449d893b958c6130869e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:09.175531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8f108-c219-4eae-9814-d28776a3193e None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:09.175531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8f108-c219-4eae-9814-d28776a3193e None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:09.175827 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8f108-c219-4eae-9814-d28776a3193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:09.203165 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-d3f8f108-c219-4eae-9814-d28776a3193e None None] MFA Rules not processed for user `f9038a6d909342fb979b2a42534ff93e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:09.204500 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-d3f8f108-c219-4eae-9814-d28776a3193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:09.270430 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1432/2882] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:09.283082 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:09.284927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:09.307705 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:09.344488 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd78a0-3dfa-4b7d-8346-734ff4277553 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:09.348235 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69dd61877f6e41ba86d2a9bb9537afc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:09.348912 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:09.349190 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:09.349471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:09.358408 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:09.359834 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:09.368720 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1451/2883] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:42:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:10.713934 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:10.714531 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:10.714680 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:10.732771 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] MFA Rules not processed for user `f9038a6d909342fb979b2a42534ff93e`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:10.734024 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:10.740971 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1433/2884] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:42:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:19.364724 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=17 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:20.726853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:26.362011 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:26.363533 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:26.436380 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580b77-88e0-47df-8232-e79097914380 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:26.441582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:26.442544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:26.443027 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:26.443471 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:26.447127 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:26.470583 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:26.472365 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:26.487747 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1452/2885] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:42:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:29.830943 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:29.832538 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:29.897931 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501e3c-13b7-47c3-a26c-4240ae050476 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:29.902690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:29.903572 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:29.903857 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:29.904134 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] PATH_INFO: `/v3/users/fc7827c5d9294b8c90f26deff983b8e3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:29.905425 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7827c5d9294b8c90f26deff983b8e3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:29.907390 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:29.961853 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7827c5d9294b8c90f26deff983b8e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:29.968666 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7827c5d9294b8c90f26deff983b8e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:29.982342 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7827c5d9294b8c90f26deff983b8e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:29.983083 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7827c5d9294b8c90f26deff983b8e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:29.992319 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7827c5d9294b8c90f26deff983b8e3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:29.999862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1434/2886] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:42:29 2022] DELETE /identity/v3/users/fc7827c5d9294b8c90f26deff983b8e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:30.178672 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:30.191248 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592a8d0-74b5-45d8-a8b2-fc43facbf1d4 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:30.251034 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:30.251894 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:30.252181 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:30.252460 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] PATH_INFO: `/v3/projects/f3f0c2dd35fa4890a64fcc7dcaddc633` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:30.253674 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f0c2dd35fa4890a64fcc7dcaddc633)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:30.255571 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:30.301160 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f0c2dd35fa4890a64fcc7dcaddc633', 'request_id': 'req-1c6167c4-3343-46b3-b989-f354838e033e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.306581 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f0c2dd35fa4890a64fcc7dcaddc633', 'request_id': 'req-1c6167c4-3343-46b3-b989-f354838e033e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.314622 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f0c2dd35fa4890a64fcc7dcaddc633', 'request_id': 'req-1c6167c4-3343-46b3-b989-f354838e033e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.327117 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] The token cache is being invalidate because project f3f0c2dd35fa4890a64fcc7dcaddc633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:42:30.327404 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.328081 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1c6167c4-3343-46b3-b989-f354838e033e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.329939 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1453/2887] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:42:30 2022] DELETE /identity/v3/projects/f3f0c2dd35fa4890a64fcc7dcaddc633 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:30.333987 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:30.335487 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:30.350217 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29e340-f8c0-4222-8e4f-71db75979542 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:30.407440 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:30.408326 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:30.408616 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:30.408920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] PATH_INFO: `/v3/users/97b9d773a0c64dc6a6c5284cf73852c3` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:30.410127 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b9d773a0c64dc6a6c5284cf73852c3)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:30.412042 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:30.435139 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:30.460499 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b9d773a0c64dc6a6c5284cf73852c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.467954 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b9d773a0c64dc6a6c5284cf73852c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.479434 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b9d773a0c64dc6a6c5284cf73852c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.480262 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b9d773a0c64dc6a6c5284cf73852c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.487458 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b9d773a0c64dc6a6c5284cf73852c3'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.493632 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1435/2888] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:42:30 2022] DELETE /identity/v3/users/97b9d773a0c64dc6a6c5284cf73852c3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:42:30.633254 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:30.633544 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:30.633823 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:30.663287 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:30.687759 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:30.689767 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:30.698313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:30.698313 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:30.771907 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea307-5377-4bf6-8034-8e708fbcad3a admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:30.773658 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1454/2889] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:42:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:30.777406 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:30.778368 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:30.778661 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:30.778941 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] PATH_INFO: `/v3/projects/cd91da4bd20948c8a4dd918ac4f88390` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:30.780148 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd91da4bd20948c8a4dd918ac4f88390)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:30.782441 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:30.825709 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:30.827979 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:30.834463 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd91da4bd20948c8a4dd918ac4f88390', 'request_id': 'req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.840920 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd91da4bd20948c8a4dd918ac4f88390', 'request_id': 'req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.849121 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd91da4bd20948c8a4dd918ac4f88390', 'request_id': 'req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.861170 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] The token cache is being invalidate because project cd91da4bd20948c8a4dd918ac4f88390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80250) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:42:30.861461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.862246 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-853f7d05-9bd4-4726-a8a7-dfbb358d42dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:30.863677 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1436/2890] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:42:30 2022] DELETE /identity/v3/projects/cd91da4bd20948c8a4dd918ac4f88390 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:30.944057 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0e5fd-4ec6-4d6f-9328-fe8d095d133c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:31.008472 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:31.009172 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:31.009454 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:31.009728 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:31.012124 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:31.029433 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:31.078624 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:31.080237 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:31.090018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1455/2891] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:42:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:42:31.614961 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29611f8-1a18-404a-b09e-693c472aaafe None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:31.615258 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29611f8-1a18-404a-b09e-693c472aaafe None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:31.615540 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29611f8-1a18-404a-b09e-693c472aaafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:31.633942 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-a29611f8-1a18-404a-b09e-693c472aaafe None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:31.635161 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-a29611f8-1a18-404a-b09e-693c472aaafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:31.644312 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1437/2892] 10.222.0.79 () {58 vars in 967 bytes} [Wed Jul 20 19:42:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:31.657397 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:31.663770 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:31.679736 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab30827-720d-4fd2-9e52-331364baa478 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:31.685345 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:31.685997 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:31.686280 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:31.686553 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:31.697668 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:31.700021 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:31.708847 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1456/2893] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:42:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:42:31.914766 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23280-fb2b-41ab-9b33-041d93c9f760 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:31.915290 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23280-fb2b-41ab-9b33-041d93c9f760 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:31.915422 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23280-fb2b-41ab-9b33-041d93c9f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:31.935149 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-fdf23280-fb2b-41ab-9b33-041d93c9f760 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:31.936839 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-fdf23280-fb2b-41ab-9b33-041d93c9f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:31.943912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1438/2894] 10.222.0.79 () {58 vars in 1009 bytes} [Wed Jul 20 19:42:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:31.956902 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:31.968816 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d3d7a-d368-4a9e-aea4-f0bcda50509b service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:31.974080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:31.974729 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:31.975080 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:31.975293 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:31.984564 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:31.986260 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:31.995024 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1457/2895] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:42:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:42:32.344285 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] REQUEST_METHOD: `POST` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:32.344583 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:32.344862 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:32.363241 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.auth.core [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] MFA Rules not processed for user `f9fd37a822f04be2ab327efb57f59b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80250) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:32.364491 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:32.372704 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1439/2896] 10.222.0.79 () {58 vars in 967 bytes} [Wed Jul 20 19:42:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:32.384570 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:32.395706 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f6e47-158d-4394-9db2-28e82c780fb8 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:32.400719 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3ef33-d4ef-44d7-814f-a97153ed801d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04be59c2cfa049629282356a9e209900', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:32.401507 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3ef33-d4ef-44d7-814f-a97153ed801d service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:32.402018 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3ef33-d4ef-44d7-814f-a97153ed801d service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:32.402303 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3ef33-d4ef-44d7-814f-a97153ed801d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:32.410761 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3ef33-d4ef-44d7-814f-a97153ed801d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:32.412335 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3ef33-d4ef-44d7-814f-a97153ed801d service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:32.421347 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1458/2897] 10.222.0.79 () {60 vars in 1384 bytes} [Wed Jul 20 19:42:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:42:34.828690 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:34.830175 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:34.841505 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:34.897178 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb57c-7a59-4e10-b42b-59919888fe79 None None] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:34.902349 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:34.903238 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:34.903522 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:34.903799 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] PATH_INFO: `/v3/users/f9038a6d909342fb979b2a42534ff93e` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:34.905057 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9038a6d909342fb979b2a42534ff93e)` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:34.906981 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:34.956830 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9038a6d909342fb979b2a42534ff93e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:34.963493 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9038a6d909342fb979b2a42534ff93e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:34.976488 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9038a6d909342fb979b2a42534ff93e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:34.977195 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9038a6d909342fb979b2a42534ff93e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:34.985882 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.notifications [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9038a6d909342fb979b2a42534ff93e'} {{(pid=80250) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:34.992135 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1440/2898] 10.222.0.79 () {58 vars in 1223 bytes} [Wed Jul 20 19:42:34 2022] DELETE /identity/v3/users/f9038a6d909342fb979b2a42534ff93e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:34.997478 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:34.997975 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:34.998093 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:35.016245 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] MFA Rules not processed for user `0b2e9296115548cda14048313e8cf43e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:42:35.017505 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:35.077264 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1459/2899] 10.222.0.79 () {58 vars in 957 bytes} [Wed Jul 20 19:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:42:35.086970 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:35.088427 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:35.146395 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe00f9-ea20-4168-9e62-128a4efc03a5 admin admin] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:35.150213 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e16aed371fa4c9da4ea02563fed64c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:35.150863 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:35.151154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:35.151431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:35.160451 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:35.161936 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:35.171912 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1441/2900] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:35.345213 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:35.346704 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:35.359627 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463da550-6ded-4758-afdc-0d0ca46d0798 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:35.364582 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b2e9296115548cda14048313e8cf43e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8afe543881644c988270ab8de48728b1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:35.365561 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:35.365927 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:35.366210 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] PATH_INFO: `/v3/projects/67f87e7aec9f4fa7a23ba9ea65d2d733` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:35.367502 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f87e7aec9f4fa7a23ba9ea65d2d733)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:35.369614 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:35.416603 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f87e7aec9f4fa7a23ba9ea65d2d733', 'request_id': 'req-ea99eb63-d846-4ede-afbd-2ab00e522a33'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:35.422714 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f87e7aec9f4fa7a23ba9ea65d2d733', 'request_id': 'req-ea99eb63-d846-4ede-afbd-2ab00e522a33'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:35.430503 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f87e7aec9f4fa7a23ba9ea65d2d733', 'request_id': 'req-ea99eb63-d846-4ede-afbd-2ab00e522a33'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:35.439439 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:35.443318 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] The token cache is being invalidate because project 67f87e7aec9f4fa7a23ba9ea65d2d733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:42:35.443789 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:35.444863 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ea99eb63-d846-4ede-afbd-2ab00e522a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:42:35.446541 dv-cinder-850548-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1460/2901] 10.222.0.79 () {58 vars in 1232 bytes} [Wed Jul 20 19:42:35 2022] DELETE /identity/v3/projects/67f87e7aec9f4fa7a23ba9ea65d2d733 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:42:45.167883 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:47:02.609555 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] Authenticating user token {{(pid=80250) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:47:02.611122 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:47:02.753608 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d80be8-d113-4468-8c3e-bae2d28ec5bb service neutron] Validating token access rules against request {{(pid=80250) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:47:02.815212 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'add72a4c345b4ab49e822f478eebd6b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec730e6d5610427eb8e3d2ae8808c123', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80250) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:47:02.815867 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] REQUEST_METHOD: `GET` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:47:02.816154 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] SCRIPT_NAME: `/identity` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:47:02.816431 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80250) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:47:02.818231 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.fernet_utils [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80250) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:47:02.829894 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:47:02.831461 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da21a2c-c203-4602-b36a-66d9c91e4547 service placement] RBAC: Authorization granted {{(pid=80250) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:47:02.841362 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: [pid: 80250|app: 0|req: 1442/2902] 10.222.0.79 () {60 vars in 1365 bytes} [Wed Jul 20 19:47:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:47:12.837125 dv-cinder-850548-1-iscsi devstack@keystone.service[80250]: DEBUG dbcounter [-] [80250] Writing DB stats keystone:SELECT=89 {{(pid=80250) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}